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Navigation and terrain familiarity are critical for mission success in the military. 
Virtual environments (VEs) have often been suggested as a useful tool in addressing 
these issues. This thesis research addresses the utility of VEs to improve spatial 
knowledge of and navigation performance through natural terrain compared to traditional 
methods. In this experiment, fifteen subjects were assigned to one of three training 
conditions. The map group studied the environment using only an orienteering map. The 
real world group studied the environment using the map and explored the actual terrain. 
The VE group studied the terrain using both the map and a real-time VE. Measures were 
taken of both route and configuration knowledge. The results suggest four conclusions. 
First, training conditions have no statistically significant effect on an individual’s ability 
to obtain and demonstrate spatial knowledge of a natural environment. Second, spatial 
ability plays a significant role in navigation performance. Third, exposure to the actual 
terrain or to a virtual representation of the terrain seems to eliminate ambiguities in an 
individual’s mental map by providing dynamic imagery to clarify propositional 
knowledge gained from maps. However, this factor has not been shown to improve 
performance by the measures used here. Fourth, a high resolution 1:5,000 orienteering 
map provides extensive detail and consequently, navigation performance in this 
experiment is not likely to be indicative of performance using a conventional 1:24,000 
map. 
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I. INTRODUCTION 



A. PROBLEM STATEMENT 

The purpose of this thesis is to evaluate if training in a real-time high fidelity VE 
is superior in terms of spatial knowledge acquisition as compared to traditional military 
navigation or orienteering training methods (Chapter n, Section B.l). The research 
establishes a benchmark from which future research will continue to define the optimal 
level of fidelity and exposure times required in a vitual environment (VE) to provide 
spatial knowledge of a complex natural environment. 

B. MOTIVATION 

1. Important Applications for Spatial Knowledge Acquisition 

Spatial knowledge acquisition is important in a variety of applications and 
professions. We use our spatial knowledge of an environment to move freely throughout 
our everyday surroundings. Traditionally, we think of people using spatial knowledge of 
VEs when playing video games to quickly move through a virtual world. However, more 
vital applications of spatial knowledge can be seen in many areas. Taxi drivers use their 
spatial knowledge to transport passengers throughout the city. Fire fighters use spatial 
knowledge to conceptualize and maneuver through the environment where they are 
working even though the area is obscured by smoke and flames. 

In fact, any task that requires movement over a distance or through a complex 
environment demands some degree of spatial knowledge. Military personnel rely on 
their spatial knowledge of the environment to perform their missions. Soldiers use spatial 
knowledge of the enviromnent to move rapidly and undetected to destroy the enemy. 
Special operations units utilize their spatial knowledge of an environment to quickly 
move through complex structures when conducting raids or hostage rescue missions. 
Other highly trained military units and personnel, such as Rangers or Scouts, use their 
spatial knowledge of an environment to conduct clandestine reconnaissance of an 
objective. They also utilize their spatial knowledge to establish ambush sites or secure 
critical terrain such as an airstrip. 
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2. Proven Concept 

Human acquisition of spatial knowledge has been studied for many years. 
Thomdyke described a simple model for how humans acquire spatial knowledge 
(Chapter II, Section A) [THOR 80]. Since then, further research has been conducted in 
many areas which support Thomdyke ’s model. In the domain of conputer science, it has 
been shown that individuals can gain spatial knowledge of VEs [GILL 97] [RUDD 98] 
using a mental model conparable to Thomdyke’ s. Similar research has indicated that 
detailed VEs can provide spatial knowledge transfer to real world environments [WITM 
95] [BLIS 97] [DARK 98]. 

3. Research Shortcomings 

Gaining spatial knowledge of a VE provides little benefit by itself. However, the 
transfer of knowledge to the real world does have many potential benefits. Individuals 
can train to perform hazardous tasks in the relatively safe surroundings of a virtual world 
prior to having to perform them in the actual setting. There are many questions about 
VEs that have not been answered in terms of how to facilitate a positive transfer of spatial 
knowledge to the natural environment. 

Computers allow us to view environments in numerous ways that are not 
physically possible in the real world. Viewpoints can be rapidly changed to provide 
individuals with alternative vantage points. We can examine the same object or location 
from any direction, plane, or altitude. We can also move through the environment at any 
speed we desire or merely teleport to the next location. However, the best combination 
of these capabilities to maximize spatial knowledge acquisition is still unknown. Studies 
focused on field of view, display devices, input devices, locomotion devices, and the 
navigation thought process are needed to find the optimal interface for acquiring and 
utilizing spatial knowledge. 

Much of the previous work has focused on increasing fidelity. However, little 
research has been conducted indicating that performance enhancement is linearly 
correlated to increasing fidelity. Few experiments involve the study of how effective 
computer based training systems are compared to prior training techniques or how to 
better use what we already have. Recently, one research project indicated that an 
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increase in environmental fidelity does not necessarily translate into an increase in 
navigational performance [GOER 98]. Research in the relationship of model fidelity to 
performance is essential to identifying the minimal and optimal levels of detail required 
to obtain spatial knowledge of a specific environment. Such research will provide 
information needed to furnish model designers and builders with a template for the 
construction of virtual worlds. 

4. Army and DoD Relevance 

Defining the levels of model fidelity and system exposure times required to 
provide a positive transfer of knowledge to the real world can save the Department of 
Defense (DoD) time, money, and other valuable resources. With the decreasing DoD 
force structure, the increasing quantity and types of missions the mihtary is asked to 
perform, and a shrinking military budget, commanders need faster methods to train their 
forces while maintaining or enhancing their abilities to successfully accomplish their 
missions. Con^uters provide commanders with the opportunity to conduct time- 
compressed training as they perform multiple mission runs changing parameters on the 
fly and reducing the wear and tear on the terrain and equipment. The major focus of 
computing and mission preparation tools should be to provide commanders with 
improved tools to maximize the quality and quantity of training that can be conducted 
during a limited time frame without introducing negative training effects. 

Virtual environments (VEs) may provide a cost-effective alternative to more 
traditional methods of training or mission preparation. The optimal conq)uter training 
system has yet to be built for use in training dismounted forces. Even if an optimal 
interface and appropriate terrain model can be built, there is still the question of cost- 
effectiveness of such systems con^ared to traditional methods of model construction. In 
the past, the military has used mock-ups or other representations of an environment for 
mission preparation [GLIN 95] [FINN 97] [AMER 98]. Although these traditional 
model building methods are effective, they can be expensive, time consuming to 
construct, difficult to modify, labor intensive, and require large areas of terrain to be 
secured during construction and mission rehearsal (Chapter II, Section B.2). 
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Little research has been conducted and published to validate the usefulness of 
VEs in providing spatial knowledge of natural terrain. If an individual can gain the same 
or only slightly better spatial knowledge using a VE over using a two-dimensional map, 
photo, or sketch of the environment, then it may not be cost-effective to utilize virtual 
representations of actual terrain or buildings to acquire spatial knowledge. It is possible 
that a VE may only be useful in training general navigational skills such as map reading, 
dead reckoning, terrain association, and route selection (Chapter II, Section C.l). Real- 
time VEs may be too complex to provide positive training transfer of spatial knowledge. 
If this is the case, a randomly generated VE may be more cost effective and just as 
efficient at training navigation skills as a virtual representation of an actual piece of 
terrain. Understanding the positive and negative effects of computer training and mission 
preparation systems will assist commanders in determining if and when they should use 
such systems; 

Understanding the actual and desired effects of computer training systems is 
essential for the development of a valid set of parameters for building models designed to 
equip users with spatial knowledge of an environment. This will assist the Army and 
DoD simulations personnel in determining when the construction and use of these models 
will be beneficial to military forces. Such standards will help to alleviate the 
misallocation of time, money, , and computer assets for missions that do not warrant the 
use of computer generated models. The parameters will also assist in providing a 
common frame of reference for which these agencies, contractors, and programmers can 
consult with each other during the development and modification of such models. 

C. THESIS ORGANIZATION 

This thesis is organized in the following manner: Chapter II explores the 
background of spatial knowledge acquisition, the use of VEs to gain spatial knowledge, 
and land navigation and orienteering techniques. Chapter III outlines the development of 
the computer model and its capabilities as well as the methodology for this experiment. 
Chapter IV analyzes the data collected and discusses the results of the experiment. 
Chapter V provides the conclusions explaining the importance of the research and 
recommended areas for future research. 
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There are seventeen appendices to this thesis that provide experimental outlines, 
listing of research materials, course layout, participant instructions, and raw data from the 
experiment. The appendices also discuss route complexities based on the International 
Specification for Orienteering Maps (ISOM) and computer generated routes, outlines 
environmental conparisons of training conditions, and describes the navigational thought 
cycle. 
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II. BACKGROUND AND PREVIOUS WORK 



A. SPATIAL KNOWLEDGE ACQUISITION 

Spatial knowledge or spatial cognition is a mental representation of a real or 
virtual environment [WICK 92]. Figure 2.1 graphically displays Thomdyke’s theory on 
how humans acquire spatial information to build a mental representation of their world 
[THOR 80]. In this model, the classifications of landmark, route, and survey knowledge 
are not mutually exclusive; knowledge at higher levels builds upon and augments 
knowledge gained from the preceding level(s). 




Exposure 






Figure 2.1. Navigation Knowledge 



Landmark knowledge is identified as the ability to recognize distinctive features 
associated with a specific location in the environment. This level of navigation 
knowledge is associated with the ability to store features, such as a specific hilltop or 
road intersection, in memory and recognizing it. Landmark knowledge is acquired 
through the direct observation of objects in the environment. It can also be gained 
through indirect observation of the objects in a medium such as a photograph. Successful 
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landmark knowledge is demonstrated by the ability to recognize individual locations or 
unique objects within an environment [DARK 95] [THOR 80]. 

Route knowledge is defined as the procedural knowledge required to navigate 
along a route or path between landmarks or distant locations [GOLL 91]. It is derived 
from the ability to expand landmark knowledge into a larger, more complex arrangement 
of linked objects. Route knowledge is based on an egocentric (inside-out) viewpoint and 
is demonstrated by the ability to move from one landmark to another along a prescribed 
path. Route knowledge can be gained through repeated exposure to an environment, 
map, or through simulated exposure to the environment via a medium such as video 
[GOLD 82]. 

Finally, survey (or configurational) knowledge is the highest level of spatial 
knowledge. It represents a map-like or top down mental encoding of the environment 
and is based on an exocentric (outside-in) viewpoint. This last form of spatial knowledge 
is usually gained through map study but, can also be gained through extensive and 
repeated exposure to the environment [THOR 80]. Survey knowledge can be 
demonstrated by an individual’s ability to describe the relative locations and the distances 
between landmarks or by devising new routes between landmarks even though the person 
has never traveled a route between them [BANK 97]. 

B. VIRTUAL ENVIRONMENTS 

1. Definitions 

Real-time graphics are defined as models which provide network delays of less 
than 0.1 seconds and can render images at a minimum of 8 to 10 frames per second for 
relatively static environments and up to 60 images per second for environments where 
objects have a high frequency of motion [DURL 95]. 

The classification of fidelity is more qualitative than quantitative since there are 
no metric scales which allow us to explicitly define levels of fidelity. For this research, 
the definition of a high fidelity walkthrough terrain model is a model that represents lines 
of sight and terrain masking, provides realistic depictions of the vegetation and structures, 
and can provide a real-time interactive environment for the user (Figures 2.2 and 2.3). 
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Figure 2.2. Computer Model Figure 2.3. Actual Photo 

Landmark models are virtual representations of real world objects or locations 
that are easily identified, with defining characteristics, and are used by the participants as 
cues to navigate through the model In Figures 2.2 and 2.3, the shack and poles are 
examples of landmark models. The term is devised for use in this thesis and derives jfrom 
Thomdyke’s theory of navigation [THOR 80]. 

2. Model Usage for Mission Planing and Preparation 

For centuries, military units have conducted rehearsals in preparation for 
missions. Rehearsals were performed at all levels of command and took on many forms. 
Commanders utilized everything from checklists, sand table briefings, walk-through 
rehearsals using soldiers and open fields, video-taped runs of air corridors, to full fledged 
dress rehearsals utilizing complete mockups of the target area. The more extensive the 
rehearsal, the more resources and time were required. The more dangerous and complex 
the mission the more rehearsals were necessary for mission success. 

During World War II, the Army Air Corps prepared its pilots for air raid missions 
over Japan by showing them films of the precise routes they would be flying over enemy 
territory. These films were not produced from satellite imagery or continuing over flights 
by American reconnaissance aircraft. Instead, the films were produced by the Air Corps 
film and production unit stationed in Hollywood, California [AMER 98]. The production 
unit built a model of the Island of Japan using over fifty ten foot square platforms, tons of 
plywood, modeling clay, burlap, and paint. Using reconnaissance photos of the island, 
crews worked twenty-four hours a day for weeks, expending thousands of man-hours, to 
construct and paint an exacting replica of the island so that camera crews could film 
bombing routes for pilots. In order to maintain security, the model was built and filmed 
entirely on a single sound stage that was placed off limits to everyone except the 
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personnel working on the project. Pilots routinely commented on how easy it was to 
recognize the terrain as they flew their missions because it was if they had been there 
before. 

In 1970, the United States Air Force planned a raid on North Vietnam to rescue 
American prisoners of war (POWs) from the Son Tay prison near Hanoi, North Vietnam 
[GLIN 95]. The mission was extremely risky. It required a task force of Air Force, 
Navy, and Army Special Forces personnel to travel over 340 miles across enemy 
territory, attack an enemy held compound, pull out the POWs, and fly back across enemy 
territory to the safety of American bases. To prepare the forces for the assault on the 
conq)ound, a replica of the prison was constructed of two-by-fours and target cloth. The 
confound was built to exact dimensions to allow helicopter crews to practice taking out 
the guard towers with side mounted machineguns and land a single aircraft in the 
conqwund for the insertion of Special Forces soldiers and the extraction of the prisoners. 
Due to the nature of the mission, security was of the utmost importance. Even though the 
camp replica was built and rehearsals were conducted in Florida, thousands of miles from 
Southeast Vsia, the model had to be dismantled daily to prevent detection by the Soviet 
Satellite Cosmos 335. Because of this, rehearsals using the mockup were conducted at 
night. Before morning, the model was dismantled and aU holes were covered to ensure 
that no information could be gathered by the satellite and passed to the North 
Vietnamese. The ground team conducted over 170 trial runs through the camp replica 
before final authorization was granted to execute the mission. All this effort required 
over 148 personnel to support and execute the operation. Although the prisoners had 
l>een removea from the prison prior to the conduct of the raid, the task force was able to 
execute the plan to exacting precision without the loss of one American life. 

The Son Tay model was constructed with the assistance of an inconspicuous 
element of the CIA known as the Modeling Shop [FINN 97]. The three-dimensional 
modeling shop was located at the National Photographic Interpretation Center, in the 
Navy Yard. The section was established in 1964 to create three-dimensional models of 
key areas of the world to help planners and decision-makers with international policy 
decisions. The section operated for over thirty years and was finally replaced by 
computer generated modeling tools in 1997. Over the thirty-three years of the shop’s 
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existence, the shop produced more than 862 products. These products include rephcas of 
the U.S. Embassy in Iran and the Iranian Foreign Ministry, the Kremlin, sections of the 
city of Tehran, Russian aircraft, a section of Kuwait City from the U.S. Embassy in 
Kuwait to the Persian Gulf shoreline, and many other U.S. embassies throughout the 
world. 

The efforts to construct detailed models and maintain security for similar missions 
requires hundreds of thousands of man-hours and an untold number of assets. However, 
the need to conduct detailed rehearsals is essential for successful execution of any 
complex operation. 

3. Prior Studies of Spatial Knowledge and Virtual Environments 

In 1997, Bliss, et al studied the role of VE technology in acquiring spatial 
knowledge as outlined in Thomdyke’s model [BLIS 97]. They asserted that the spatial 
knowledge gained by navigating through a VE was comparable to the knowledge gained 
by navigating through the actual environment. They examined 30 firefighters performing 
simulated rescue operations in an office building. The firefighters were broken down into 
three study groups. The first group was given a map of the building to study, the second 
group was provided a VE, and the final group, the control group, was sent into the 
environment with no prior training. As they predicted, the VE and map study groups out 
performed the control group in the task of navigating through the structure. The 
researchers concluded that landmark, route, and survey knowledge can be acquired 
through the use of a VE or map. Although the Bliss experiment’s performance measures 
indicated that VEs can provide landmark and route knowledge, there were no 
performance measures nor any qualitative analysis which suggested survey knowledge 
could be attained from VEs. 

Bhss’ research did not show any significant difference in performance between 
the VE group and the map only group. This begs the question of why we should use a 
VE to obtain spatial knowledge if a map provides the same or similar comprehension. 
This question is best addressed by a study conducted by Chase in 1983 [CHAS 83]. 
Chase examined individuals who studied maps of an environment which they had never 
been exposed to, and a second group of individuals who had extensive exposure to the 
environment but, never studied a map. His research suggested that the individuals with 
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extensive exposure to the environment had better landmark and route knowledge. 
However, the map study group had better survey knowledge of the environment. 
Therefore, Chase concluded that repeated exposure to an environment provides route and 
landmark knowledge but, this experience does not necessarily translate into increased 
survey knowledge. As stated earlier, survey knowledge is often obtained through 
exposure to an exocentric view while route and landmark knowledge are normally gained 
through repeated exposure to the environment. 

Wickens and Prevett showed that aviators who were exposed to an “immersive” 
viewpoint had better navigational performance over those who were given tethered or 
side views [WICK 95]. This is possibly due to the more natural representation or 
intuitive viewer interpretation based on the field of view [WICK 98]. Taking this into 
account along with the Chase experiment, it is possible that the combination of map study 
and VE exposure may provide the optimal solution for providing total spatial knowledge 
of an environment [CHAS 83]. 

Banker showed that individuals with intermediate land navigation skills could 
gain and transfer their spatial knowledge firom a VE to a natural environment [BANK 
97]. He studied three groups; a map only group, a map and non real-time VE group, and 
a control group who studied a map and the actual environment. None of the participants 
had any prior knowledge of the course, terrain, or map. Participants were give one hour 
to study the environment and plot their route through the environment before the map 
was taken away from them and they were required to navigate the real world course. 
With only a one-hour exposure time. Banker discovered that the VE showed a significant 
increase in performance for intermediate navigators only. He concluded that due to 
advanced navigators’ abilities to extract vast amounts of information fi-om the map, the 
VE was of no additional advantages to them. He further surmised that beginning 
navigators had reached information overload with all the materials provided and they 
were unable to separate the noise fi'om the essential information. 

Goerger, et al conducted a similar study to the Banker experiment using a 
complex man made environment [GOER 98]. They compared two groups; a map only 
study group to a map and VE study group. None of the participants had any prior 
exposure to the building or the floor plans. Each group was given thirty minutes to study 
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floor plans of the seven-story structure and a clue sheet (Figure 3.17). The VE group was 
also exposed to a high fidelity, real-time conputer representation of the building during 
the thirty-minute study phase. The floor plans had the control points and a designated 
route marked in red. After the thirty-minute study phase, the floor plans were taken firom 
the participants who were then escorted to the building for the testing phase. The results 
of the experiment showed the map only group significantly outperformed the VE group. 
The researchers concluded this was due to the short exposure time. They surmised the 
limited exposure time did not allow the VE group to resolve the exocentric differences 
between the - floor plans and the virtual world. Furthermore, this did not allow 
participants to translate their knowledge of the environment fi-om the maps to the VE to 
the real world. This indicates that performance on spatial knowledge tasks after brief 
exposure to a high fidelity, real-time VE does not always exceed results gained firom 
traditional navigation training techniques [GOER 98]. 

Although it has been demonstrated that virtual worlds can be effective in 
providing some level of spatial knowledge, we are far from understanding which 
characteristics of a VE and its interface are most effective. A study by Witmer, et al 
investigates the issue of user interface [WITM 95]. They examined 64 college students 
navigating a large building. The participants were broken into three groups; those who 
received verbal directions on how to navigate through the building, those who explored a 
VE of the building, and those who explored the actual building. Results indicated that 
the real world participants outperformed the VE participants who in turn did better than 
the verbal study students. Although the VE group outperformed the verbal group, some 
VE participants did not perform well. Further analysis of the training data indicated that 
VE participants who became entangled in the model due to difficulties with the interface 
spent much of their time bouncing into walls, becoming disoriented, and subsequently 
made many wrong turns. These same individuals had difficulty navigating through the 
actual building. Although these are qualitative measures, the data indicates that a poor 
interface can lead to disorientation and diminish the overall training efficacy of the VE. 

In other research, Williams, et al investigated active versus passive control during 
flight mission preparation [WILL 95]. The study compared individuals who passively 
observed a flight through a virtual world to those who actively controlled the flight of the 
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aircraft through the high fidelity flight simulator. The study indicated that those 
participants who controlled the rehearsal phase outperformed those who passively viewed 
the training flight. They concluded that the optimal VE designed to enhance spatial 
awareness would include active control by the user. 

Based on these studies we can conclude that VEs are useful in providing 
landmark and route knowledge. Augmented with map study to provide survey 
knowledge, the two media can provide a powerful method for obtaining spatial 
knowledge of unfamiliar environments. To optimize the use of a VE, the interface must 
be transparent to the user while providing active control of the viewpoint. Optimal 
exposure durations and fidelity levels are still a questionable element of the VE. 

4. Model Classiflcations 

The first aspect that must be considered before building any model is its purpose. 
Requirements for building a three-dimensional architect’s rendition of a building’s 
blueprints and the requirements for building a real-time walk through model of the same 
structure are significantly different. The first may require only a wire frame or flat 
shaded polygons. The latter may require expanded coloring schemes and texturing to 
provide a realistic appearance. The architect’s rendition may not be concerned with 
collision detection, the portrayal of furniture, sound effects, or independent moving 
entities in its VE, while the real-time walkthrough model may. All these factors 
determine the system requirements, the type and quantity of environment ' iormation 
required, the programming language, the tools, the file structure, and the procedural steps 
for the construction of the model. 

Virtual environments have developed beyond the initial desert and overfligti. 
models produced in the early 1970’s through the 1980’s for the Department of Defense. 
Today’s models have extended into the realm of complex natural environments. 
Throughout this thesis, the term “complex natural environment’’ is utilized. This 
classification of topography represents a piece of terrain at least one square kilometer in 
size which consists of vegetation and elevation changes that mask lines of sight and 
provide obstacles to cross country movement. To facilitate navigation, the terrain must 
have numerous distinctive landmarks. Finally, it possesses numerous paths, trails, or 
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roads that provide opportunities for parallel navigation errors (Chapter IV, Section 
A.4.a). 

Terrain models can be separated into four distinct categories based on the mode of 
travel through the environment. The four model categories are Dismounted Movement, 
Ground Vehicles, Rotary Wing Aircraft, and Fixed Wing Aircraft (Figure 2.4) 
[SULL 98]. Each of these categories requires a lucid level of fidelity or combinations of 
levels-of-detail (LODs) that must be considered before the start of model construction. 




Figure 2.4. Fidelity vs Mode of Travel 

The first category is the Fixed Wing Aircraft Model. Jet aircraft simulators and overhead 
“God’s eye” view are in this category of model that requires limited detail due to the 
speed and altitude of the viewer. Models of this category typically consist of aerial 
photos or satellite imagery placed over elevation data. Distinguishing landmarks such as 
cities, lakes, major roads, valleys, and large landmasses facilitate navigation through this 
type of environment. 
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The Rotary Wing Aircraft Model is used in helicopter trainers and “pop-up” 
views. This model requires enough detail to distinguish groups of trees and buildings, as 
speeds are reduced to less than 100-mph and altitudes less then 200 feet. A model of this 
type wUl combine aspects of models from the higher resolution category Ground Vehicle 
Models with those of the lower resolution Fixed Wing Aircraft Model. Levels-of-detail 
(Chapter IQ, Section C) are used to smoothly page models in and out as the point of view 
moves between modes of travel. Navigating through this type of environment is 
facilitated by the user’s increased ability to distinguish landmarks such as road and river 
intersections, small lakes, city blocks, draws, ridges, and hill masses. Additional items 
such as power and telephone lines, side roads, and major landmarks must be modeled in 
this environment. This increased fidelity assists in navigation and in providing the detail 
required allowing the aircraft to maneuver through the terrain in a realistic manner. 

Surface level perspectives begin with the Ground Vehicle Model. Lines of sight 
must be preserved in order to provide the illusion the viewer in moving through the 
terrain. Realistic masking and unmasking of natural and man made landmarks are key to 
providing a representative model. This model also requires a higher level of detail as 
actual three-dimensional objects, such as buildings, statues, bridges, and signs are 
represented and distinguishable to the degree where they can be utilized as landmarks and 
navigational aids. General textures may be utilized to provide increased detail to the 
buildings and terrain providing a more realistic illustration. At speeds greater than ten 
miles an hour, texture fidelity can be reduced since actual details are blurred due to the 
velocity of the camera. This is based on the understanding that as views are overloaded 
with information or exposed to a high-density display, they filter the scene based on 
global and local perception [OLSO 70]. In such an environment, the individual will 
focus on objects which are unique or out of place with regard to the surrounding 
environment [ERIE 81]. 

The last category is the Dismounted Movement Model This category requires 
the greatest level of detail of the four. The limited speeds of the individual moving 
through the model and the approximation of the individual to the objects in the model are 
the primary factors for increased resolution. In a fly or drive through scenario, the level 
of the trees and buildings, or the boundaries of roads, trails and clearings are the 
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customary limits of the camera. In a walkthrough environment, the user may need to 
infiltrate through the low ground or riverbeds, seek concealment in a tree line, pursue 
cover in a ditch or trench, or setup a patrol base in a thicket. Thus, much more 
knowledge of the actual environment is required £ind a greater level of detail must be 
represented. This is the foundation on which the model for this experiment was 
developed (Chapter HI, Section C.). 

Due to the general nature of the categories, the ever-increasing capabilities of 
computer hardware, and the expanding demands of the user, the borders between these 
categories are blurred and in many cases overlap. Further research must be conducted to 
sohdify these boundaries and estabhsh a set of general specifications to assist model 
developers and clients in determining what type of model is required. This can only be 
done through human experimentation. As computers and the virtual modeling 
community continue to move forward, the boundaries between these categories need not 
continue to shift towards increased fidelity requirements. This model can be sohdified if 
category specifications are standardized based on complete understanding of user task 
requirements. 

C. LAND NAVIGATION AND ORIENTEERING 

1. Military Land Navigation 

Efficient, well directed navigation is the process of moving through an 
environment in a manner in which the individual knows the start position, current 
position, destination, route to travel, and distance traversed. Navigation requires 
knowledge of location, direction, and destination, while having a means of travel through 
the environment [WICK 92]. It is also an evolving process. Navigation not only 
involves acquiring knowledge of and strategies to move through a space, it also requires 
modification of this metaknowledge of the environment as we move through the space 
and identify changes or inconsistencies with our mental representation [JUL 97]. 
Navigation can occur in, on, or through many different media such as land, sea, air, and 
space. 

For military personnel, navigation plays a fundamental role in nearly any mission. 
For Marine and Army personnel, this is even more apparent. The Department of the 
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Army (DA) has recognized the importance of map reading and navigation skills for its 
personnel and has woven training requirements into its latest doctrine [FM21 93]. 

The Army implements a building block approach to training its people in 
navigation. Training starts at the initial indoctrination of personnel and continues at unit 
and Department of the Army training facilities. As soldiers move up the chain of 
command, the level of required expertise in navigation and map reading skills increases. 
Therefore, the DA has initiated land navigation in its basic training courses to ensure 
soldiers have the minimal skills for basic map reading and dead reckoning. Additional 
education occurs at the initial and intermediate leadership courses to refine and enhance 
navigation competence. 

With a basic proficiency of map reading and dead reckoning, the Army feels their 
soldiers have the navigational foundation required to perform as team members during 
cross-country movement. However, military leaders require additional training that 
focuses on route selection and tactical movement techniques. These skills are trained at 
the unit and during basic leadership courses. Finally, the military also implements a more 
intense level of navigation and map reading instruction at the intermediate leadership and 
staff schools. This final level of navigation training is designed to ensure that mid level 
leaders and staff officers have the increased abilities to conduct detailed analysis of an 
area of operations. These personnel are expected to glean information from military 
maps, non-military maps, and aerial photos to develop an accurate and timely analysis of 
any area. 

Basic skills taught in military land navigation training include map reading and 
use of an M2 or lensatic conpass. Other navigational tools and field expedient methods 
for determining cardinal directions such as watch, star, and shadow-tip are also taught. 
Even the use of a global positioning system (GPS) for determining location has become 
an essential part of the curriculum. For cross-country movement, the military focuses on 
two basic land navigation techniques; dead reckoning and terrain association. These 
skills are perishable and must be continuously practiced to maintain proficiency. 

Dead reckoning is the ability to navigate through terrain without the use of trails 
or intermediate landmarks. It consists of two fundamental steps. The first step consists 
of determining the direction of movement and the distance to travel. The second step 
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incorporates traversing the terrain utilizing a mechanism to deteirnine direction of 
movement and a device to record the distance covered. This technique of navigation is 
normally associated with dismounted cross-country travel during limited visibility or 
through thickly vegetated environments. 

The most difficult and valuable capability in military navigation is terrain 
association. This skill is used to visualize the features of the map and correlate them with 
the actual terrain features. The process of terrain association starts with aligning the map 
to the terrain. The second step is to determine one’s location. This step is key to 
successful navigation. With knowledge of one’s current location, a navigator can then 
determine the distance and direction to the destination. Without knowledge of one’s 
location, the individual is doomed to roam aimlessly around the terrain until a fix is 
obtained. 

Terrain association is also more forgiving than dead reckoning. Errors from 
terrain association can easily be resolved as the navigator reestablishes location and 
adjusts direction of travel. Errors with dead reckoning may not be discovered until the 
navigator has reached the prescribed distance. Then the actual location must be 
ascertained before a new direction of travel and distance to the intended destination can 
be determined. While utilizing terrain association, a navigator will focus on prominent 
terrain features, such as hilltops to guide direction of travel. The navigator may also use 
handrails, such as rivers or ridgelines to guide movement. Handrails are linear terrain 
features such as a river, road, trail, power line, or ridge that is paraUel or congruent with 
the desired or most direct route that an individual follows as a guide [FM21 93]. 

Finally, to prevent over shooting the objective, a navigator utilizing terrain 
association will use a catching feature, such as a stream, to act as a limit of advance, 
stopping movement in a particular direction. A catching feature is a prominent piece of 
terrain used by navigators to indicate to the need to change direction or stop movement 
[FM21 93]. 

To assist with training and honing navigational skills, the military runs land 
navigation and orienteering courses. Orienteering is a competitive sport that combines 
land navigation and cross-country running. The Army divides orienteering into four 
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categories; route orienteering, line orienteering, cross-country orienteering, and score 
orienteering. Route orienteering is used for novice navigators. Soldiers follow behind a 
guide who takes them through the course. Soldiers are required to trace the route on their 
maps and circle the location of control points located along the route. Line orienteering 
consists of copying a pre-designated route onto one’s own map. Soldiers then follow the 
prescribed route and circle the location of controls located along the route. At the 
completion of a route or line course, m^s are compared to a master map for accuracy. 
Cross-country and score orienteering are two of the most common forms of what is 
known m the broader category of sport orienteering. 

2. Sport Orienteering 

Sport orienteering involves navigating through an environment utilizing a map, 
clue sheet (Figure 3.10), score card, and compass in order to find a series of three sided 
markers known as controls. Each face of the control marker is usually 12” by 12” and 
colored half international orange and half white (Figure 2.5). Barring no obstacles to 
observation, control point markers are designed to provide a participant with a 
recognizable view no matter which direction the control point is approached [LOWR 89]. 




Figure 2.5. Control Marker 

Most orienteering events are established in natural environments with multiple 
courses that provide varying degrees of difficulty. The length and the location of the 
control points determine the difficulty level of a course. The further apart and more 
technically demanding the placement of the control points, the higher the overall rating of 
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the course. Course levels are broken down by ability groups from beginner, advanced 
beginner, intermediate, and advanced competitors and are identified by a color-rating 
scheme. A white course, or beginner course, is typically 3km straight-line distance and is 
conprised of anywhere from seven to fifteen control points. An orange course, or 
advanced beginner course, is usuafty slightly longer with control points placed in more 
restrictive terrain than the white course. 

Contestants are allowed to plan their own routes through the course based on their 
skills and experience. Some courses require competitors to find the controls in order 
(cross-country orienteering) while others allow contestants to find the controls in any 
order the participant desires (score orienteering) [FM21 93]. A circle on the map 
indicates the locations of the controls. The circles normally cover a 30 to 50 meter area 
and are further defined by descriptions on a clue sheet. An individual proves that they 
have visited a control by “punching” his or her score card with a punch which produces a 
distinctive pattern on the score card [BANK 97]. A combination of shortest time and 
number of control points found score the event. Each control point has a weight assigned 
to it based on its technical difficulty. The individual with the highest point total and 
fastest finish time is the winner. 

Land navigation training affords researchers the ability to provide an adequate 
level of spatial knowledge through the use of proven training methods while 
implementing new training devices. Sport orienteering furnishes the testing platform for 
studying the spatial knowledge of individuals who have undergone land navigation 
training. The combination of the two makes for an excellent research opportunity. 
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m. METHODOLOGY 



A. TEST ENVIRONMENT 

A combination of military navigation and sports orienteering was used in the 
development of the methodology of this experiment. The course length, difficulty level, 
clue sheet, and marking system are comparable to an average orienteering orange course 
(Chapter II, Section C.2). The use of terminology, reliance on memory skills, and 
navigation techniques are more closely related to military land navigation. 

The orienteering course is established on a 1.2km by .7km piece of terrain.’ The 
terrain is located on the central coast of California in the former training area of the 
recently closed Fort Ord. The terrain is populated with outhouse facilities, shacks, 
pavilions, telephone lines, and a criss-crosseded trail and road network. Elevation varies 
from 90m to 123.4m above sea level. The limited yet distinctive changes in elevation 
provided a course that was neither a test of athletic ability nor a flat featureless 
environment. 

The vegetation on the teirain can be broken down into three distinct categories; 
perennial grasslands, oak forest, and maritime chaparral. Perennial grasslands cover 
about one fifth of the course and are characterized by knee to waist high grasses and 
some widely scattered oak trees and underbrush. These areas have excellent visibility 
and possess limited obstacles to cross-country movement. The second category is the oak 
forest that makes up nearly forty percent of the course. This terrain is populated with 
inland or coast live oaks that vary in height from 25 to 45 feet. Inland oaks often have 
canopies that reach all the way to the ground creating mobility as well as visibility 
obstacles. The woodland varies in density and undergrowth allowing some areas to be 
easily traversed. Other areas are thick with vegetation or contain large quantities of 
poison oak making cross-country traversal more difficult. 

The final category is maritime chaparral, which makes up forty percent of the 
course. Chaparral tends to grow in dense uniform thickets and has abrasive 
characteristics which create a considerable barrier to cross-country movement. Since 
cross-country movement through these areas is not recommended, walking around, as 

’ The dimensions of New York City’s Central Park are approximately 0.9km by 4.2km which is 
four times the size of this model [MAP 98]. 
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opposed to attenpting to battle through, is the best way to negotiate these areas. 

Due to its use as an active duty Army installation for much of this century, the 
area is criss-crosseded with a network of paths, trails, and minor roads. In conjunction 
with the occasional man-made structures listed above, the trails allow for the creation of a 
course rich in landmarks with many opportunities for the navigator to make parallel and 
mirror errors [BANK 97]. Parallel errors occur when individuals are not on their planned 
route or desired location but, on a route that runs parallel to the intended route or similar 
location within the environment. This occurs because of the ambiguities of the 
environment, which allow individuals to make mistakes without realizing their errors. A 
mirror error happens when a participant reaches a decision point, is faced with multiple 
options, and chooses the wrong one. For exanple, the participant comes to a fork in the 
road where he originally planed to take the right fork but, instead takes the left fork. 

B. MAP AND COURSE DEVELOPMENT 

MAJ William Banker utilizing the ISOM [INTE 90] developed the course map 
(Appendix F.3) [BANK 97]. Banker used a 1993 digitized aerial photograph of Fort Ord 
to provide the general outline of the environment. He also conducted ground 
reconnaissance to categorize the terrain and identify features not visible in the photo. 
The map was field checked by MAJ Banker in May of 1997 and by CPT Goerger in May 
of 1998. The scale of the map is 1:5,000. Traditional military operations maps use 
1:25 ,(XK) or 1:50,(XX) scale maps. Conpetition orienteering maps are usually produced at 
a scale of 1:15,000. The higher resolution map for this course provided an extremely 
accurate depiction of the terrain. Trails and features down to two meters in diameter 
could be represented on this map. This afforded participants numerous landmarks and 
paths from which to plan their routes. The unconventional scale of the map may cause 
problems with discerning distance while the orienteering symbols and terrain features 
may cause participants unfamiliar with such terminology some confusion. To assist 
participants in overcoming these problems, a legend describing each symbol and color 
code as well as a distance scale are attached to the top of the map. 

The course begins at the intersection of Gigling and Watkin’s Gate roads and 
extends to the southwest. It consists of a starting point and nine control points. The 
control points were placed in accordance with the standards for a traditional orienteering 
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orange course. This means none of the control points are located on trails or roads. This 
ensures that participants will not merely stumble across control points and requires all 
participants to conduct some cross-country movement. The placement of control points 
allows for numerous routes to each location while limiting the possibilities for 
revisitation of previously traveled routes [BANK 97]. No extraneous or false control 
points are located on the course. 

Participants are required to navigate through the control points in order. The 
straight-line distance between the control points is 2070m. Planning a very conservative 
route which sticks as close as possible to the roads and trails, a participant can plan a 
route in excess of 4560m or 2.85 miles. The course is designed to test memory and 
navigation skills while limiting the physical skills required to complete the course. 

C. MODEL DEVELOPMENT 

The model is a real time replica of the test environment and was developed based 
on the aerial photograph, course map, and ground reconnaissance. It was created on a 
SiliconGraphics Industry’s (SGI) Onyx Reality Engine-2 workstation (Table 3.1). 



Parameter 


Value 


Machine Type 


SGI Onyx Infinite Reality 


# Processors 


4 


Processor Speed 


194 MHz n>25 


Processor Type (CPU) 


MIPS RIOOOO; Chip Revision 2.5 


Processor Type (FPU) 


MIPS RIOOIO; Chip Revision 0.0 


Main Memory 


256 Mbytes 


Texture Memory 


4 Mbytes 


Graphics Pipe 


RealityEnginell 



Table 3.1. Machine Characteristics 



While moving through the model at six to ten miles an hour, the system generates • 
34,360 plus triangles at lOHz utilizing only one processor.^ The model was developed 
on an SGI graphics workstation in anticipation that within two to three years, graphics 

^ The terrain model with all attachments consist of over 50,850 polygons (18,336 terrain polygons; 
14,690 billboards at two polygons each; 65 structure models at an average of forty polygons each; two 
vehicles at 50 polygons each; 77 forest walls at two polygons each; 144 (+) roads, trails, paths, & trendi 
line sections at two polygons each). 
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workstations costing ten to twenty thousand dollars and possible high end personal 
computers will be able to run a model of similar complexity at con 5 )arable rates of 
performance. 

OpenGL and IrisGL are the substructure languages for some comprehensive 
development tools such as Multigen and Corypheaus's Designer Workbench, 
EasyTerrain, and EasyScene. A combination of Multigen and the Corypheaus tools were 
used in the development of the model. 

Dismounted movement models [SULL 98], which replicate cross-country 
movement through rough terrain, require elevation accuracy to the nearest meter or better 
to represent ditches and holes. Model elevations are acquired from Digital Terrain 
Elevation Data (DTED) repositories. The experiment’s model consists of DTED-2 data 
modified using Microsoft Excel to create DTED-5, one elevation post every meter. As 
part of his research in model building in 1997, MAJ William Banker made these 
modifications. 

DTED is produced and distributed by the Department of Defense's National 
Imagery and Mapping Agency (NIMA) and comes in Level 0 through Level 5. DTED 
Level 1 (DTED-1) is one elevation post for every 100 meters and Level 2 DTED (DTED- 
2) is one elevation post for every 30 meters. Both DTED-1 and DTED-2 are 90 percent 
accurate to +/- 30 meters. Levels 3 - 5 are classified and only available to qualifying 
agencies and personnel. DTED-1 h, available to the general public for the continental 
U.S., Alaska, Hawaii, Puerto Rico, and the Virgin Islands [EART 95]. 

Levels-of-detail (LODs) are utilized to specify at what resolution a section of 
terrain is rendered. When a model is created, sectors are established to help with the 
computer's culling process by identify what items should be displayed with respect to the 
location of the viewpoint. Sectors are established in a database as rectangular sections of 
terrain where the size of the sectors is based on the distances established for the LODs. 
These sectors are tiled together to create a grid. The sector where the viewpoint is 
located is displayed at the highest level of resolution and adjacent sectors are displayed at 
the next level of resolution. The further the sector is away from the point of view, the 
lower its level of resolution (Figure 3.1). 
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♦ - Point of View (POV) 



Di - Radius of Sector 

D 2 - Distance ifrom the 
POV to the Sector’s 
center 



Figure 3.1. LOD Sector Grid 

As the viewpoint moves through the grid, sector resolution levels will switch to 
accommodate the new viewpoint position [VOGT 97]. Adjacent sectors are switched to 
the highest level of resolution when the distance between the point of view and the center 
of mass of the next sector (D 2 ) is equal to, or smaller than, the distance between the 
center of mass of a sector and one of its comers (Di). The distance for LODs was 
determined utilizing the formula for the resolution angle of an object. This angle is based 
on an object’s distance from the participant thus, [distance to the object = (size of the 

object / 2) / tan (resolution angle / 2)] [SCHI 82] [SPER 97].^ On angles less then 10°, 
Schiffman asserts that the visual angle does not need to be divided in half. The revised 
equation is [distance to the object = (size of the object) / tan (resolution angle)]. 

O'Kane illustrated the stages of target acquisition based on Johnson's bar pattern 
methodology [OKAN 95] [JOHN 58]. She describes how a potential target goes through 
four phases: find, detect, recognize, and identify. The last three phases have application 
in the graphics world. Detection is when an object can be discerned from its background, 
for example; there is an object on the hill. As we draw closer to the object it is 
recognized as its features are discemable enough to place it in a specific category; i.e. the 

^ Schifhnan explains how to determine the visual angle of an object based on the size of and 
distance to the object. He uses the formula [tan (visual angle/2) = (size/(distance/2))]. Sperber states in his 
notes that the resolution angle, the angle at which two similar items are no longer distinguishable as 
different objects, is l/bO* of a degree or less. 
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object on the hill is a truck. Finally, we approach close enough to the object that we can 
identily it by resolving its unique features; i.e. the truck is a Ford FI 50. The closer we 
get to the object the greater the size of the object’s visible angle is in our field of view. 
These distances and angles can be calculated using Johnson's bar pattern methodology 
[JOHN 58]. For more common military targets, the angles for detection, recognition, and 
identity have been calculated using Johnson's bar pattern method and are listed in mrads'* * 
in Dudzik's Electro-Optical Systems Design, Analysis, and Testing [DUDZ 93]. 

Utilizing the above formula and Dudzik's table for resolution factors for a soldier, 
distances were calculated for identification, recognition, and detection distance. The 
average soldier standing in open terrain on a clear day can be identified up to 234m. The 
same soldier is recognized as a human up to 492m away and can be detected at distances 
up to 1247m with the naked eye.^ 

The model has three LODs. The first change over point is placed at 150m, the 
second at 450m, and the final LOD extends out to infinity. If the model were larger, the 
third LOD change over point could have been placed at 2000m or the far clipping plane 
could be set at this limit. The LODs for this model were based on the abilities of the 
computer system to handle the model fluidly, the previously stated calculations, and the 
standard distances used in Army marksmanship. As discussed earlier, individual target 
silhouettes lose their identity at 230m. Humans are unrecognizable at 450m to 500m and 
are not detectable at distances greater than 1250m when viewed by the naked eye. The 
average rifleman classifies targets within 100m as close targets. Starting at 150m, targets 
are considered midrange [FM23 89]. Army sniper training routinely works within the 
1000m range even with a lOx scope [FM23 94].^ Utilizing the 450m (diagonal 636m) 
mark as the limit of recognition of a human, this was established as the switching point 
for the second and third LODs. Attempting to maintain square sectors and using the max 
range of close targets, the switching point for the first and second LODs was placed at 

1 mrad equals 5.72958 x 10'^ degrees [BEYE 84]. 

’ The same relative distances can be calculated using a combination of Schiffman’s formula [SCHI 
82], Sperber’s resolution angle, and the MCLSTD-1472D’s [SPER 89] 95*'' percentile measurements. For 
the average adult male’s interpupillar breadth (7.1cm), head height (14.5cm), and chest breadth (36.7cm), 
the average individual will not be able to distinguish a pair of human eyes at 244m, a head on a pair of 
shoulders at 498m, and the human stature at 1261m. 

* When estimating range, a sniper will determine the distance to a 6-foot man measuring two mils 
as lOOOm [FM23 94]. 
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150m (diagonal 212m) (Figure 3.2). This created sectors which were 300m x 300m and 
placed the switching point for the last LOD at 750m (diagonal 1060m). 




450m 



I 7S0m I 

I 2000m I 

Figure 3.2. LOD Distances 

Once the digital terrain data was converted into a terrain model and LODs, the 
model's surface was colored with earth tones reflective of the actual terrain. A black and 
white aerial photo texture was then attached to the terrain (Figure 3.3). This texture map 
was used to place roads and buildings and to define the boundaries of the wooded areas. 
The black and white photo was left on the final model to emulate shadow effects. The 
black and white photo allowed the base color of the model to bleed through the photo and 
give the terrain a more natural appearance. 




Figure 3.3. Elevation Model with Black and White Aerial Photo 
After the terrain base was generated, landmark models were constructed and used 
to populate the terrain's surface. Landmark models are replications of structures that are 
distinctive and easily identifiable in the environment. Using photos and measurements of 
the structure, reahstic replicas of the structure's exterior were developed. Telephone 
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poles, two small shacks, five individual outhouses, two distinct pavilions, two distinct 
ypes of cement pads, three separate rock piles, and sandbags were developed and used as 
landmark models. The aerial photo helped maintain alignment and object orientation 
when placing landmark models onto the terrain model. 

Next, linear objects such as roads, trails, ditches, and power lines were placed. 
These linear objects helped establish boundaries and preserve proportions. With 
boundaries clearly defined, vegetation was added. Since most vegetation is con^lex in 
structure and liberally distributed, billboard textures (Figures 3.4 & 3.5) were utilized. 
The use of billboards greatly reduced the computational requirements for rendering the 
wooded areas. The billboards rotate with the movement of the camera, always providing 
the user with a perpendicular view of the object. Billboards provide the illusion of three- 
dimensional objects. 





Figure 3.4. Textured Billboards Figure 3.5. Untextured BDlboards 

Four distinct trees, three types of under brush, and two bushes were utilized in the 
model (Figure 3.6). An essential aspect of creating realistic billboards is the use of 
textures. Editing textures is more of an art than a science. Although there are many 
techniques for editing images, a commonly used methodology is to adjust colors and light 
levels, alter perspectives, correct image impurities, crop, size, save in an appropriate 
format, and final editing. These steps were completed utilizing Adobe Photoshop and 
MultiGen. The same textures used in the construction of the brush billboards were used 
in the development of walls that were intermixed with the billboards to provide the 
appearance of a forest. 
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Figure 3.6. Trees, Bushes, and Walls 

The four tree billboards were randomly placed based on their size, largest to 
smallest, at a ratio of 40, 30, 20, and 10 percent respectively. The three distinct types of 
brush and two bushes were evenly mixed at 20 percent each. The three brush textures 
were also used to create three distinct wall types to represent forests at distances greater 
then 450 meters. A thick forest, approximately 25 percent of the model, has a dispersal 
of 15,000 trees and 10,000 undergrowth billboards per square kilometer. Moderate 
forests, 20 percent of the model, had 15,000 trees and 7,000 pieces of undergrowth per 
square kilometer. Forests with little to no undergrowth make up nearly 35 percent of the 
model and are covered with 7,000 trees and 500 pieces of undergrowth per square 
kilometer. About 10 percent of the model is covered with undergrowth at 10,000 plants 
per square kilometer. The rest of the terrain is a mixture of trails and open ground 
(Figure 3.7). In total, the model uses 34 textures and 9 billboards to populate the 
environment. The final model contains: twenty-two telephone poles, two shacks, five 
outhouses, three pavilions, nine cement pads, three rock piles, two sand traps, a paved 
road, over 9,960m of dirt roads, trails, and paths, two trench lines, 200 plus sandbags, 77 
forest waUs, and over 14,690 billboard trees, bushes, and brush. Two military trucks, 
(HMMWV) were also added to the model. These vehicles were used to provide a 
common object as a reference to resolve issues of size and perspective (Chapter IV, 
Section B. 12). 
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Figure 3.7. Model Populated with Objects 
The LODs were determined for movement along the ground at a camera height of 
five feet eight inches. However, if a participant decides to travel along the terrain at an 
elevated level, he will experience a “popping” in and out of forest walls (Figure 3.8) and 
trees (Figure 3.9) as he crosses an LOD switch over point. Programmers use LODs to 
render closer objects at a level of higher detail than objects at greater distance. This is 
done by instructing the system to swap in images and structures with greater detail for 
closer objects and structures of lesser detail for further objects. 




Figure 3.8. Wall Sketch 




Figure 3.9. Trees Sketch 
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Depending on the distance and height of the viewpoint, the exchange of walls for 
independent billboard is visible and is commonly referred to as “popping”. To aUeviate 
this popping, the model may require dynamic LODs that change depending on the 
camera’s distance above the ground. Additional LODs can be added which allow the 
modeler to slowly thin the trees out at different distances and makes the popping more 
understated. These options can be computationally expensive. To reduce model 
conq)lexity, these features were not implemented in this model. 

When viewing forest walls from an elevated position, the user can gain a false 
sense of vegetation density because of the lack of vegetation behind the walls (Figure 
3.10). Placing a canopy over the top of the forest walls to represent wooded terrain can 
curtail this misperception. To reduce the amount of texture memory used for this model, 
forest canopies were not implemented in this model. 




Figure 3.10. Elevated Wall Sketch 

Once the terrain, structures, and vegetation were assembled and in place, the basic 
model was complete and the interface was developed. Utilizing the Coryphaeus 
EasyScene and API tools, an interface was developed to allow the model to be explored 
and the environmental conditions to be modified (Appendix P). Utilizing the BG 
Systems FlyBox (Figure 3.1 1) and a standard 124 key keyboard, participants were able to 
move through the environment at speeds of up to 10 miles an hour with viewpoints fixed 
at five feet eight inches or fifteen meters above the ground. Environmental conditions 
could be set for one of six conditions: sunny, cloudy, stormy, dawn, dusk, or midnight. 



33 




Figure 3. 1 1. BG Systems FlyBox 

The BG System FlyBox joystick is used as the main interface because it provides 
the participant with a compact set of instruments that can be manipulated utilizing only 
two hands without rec..iring an individual to have the dexterity required to use a 
keyboard. The joystick provides six degrees of freedom. This allows the participant to 
maneuver the viewpoint forward and back using a side lever and orient the camera’s pitch 
and rotation as well as the movement heading using the joystick. The automatic 
centering feature of the joystick ensures the aligning of an individual’s field of view with 
the direction of movement when a participant releases the controls. 

Participants can teleport the viewpoint to any of the nine checkpoints. If 
navigational assistance is needed the interface allows the user to have a top down view of 
the model from 1500 meters or a compass is displayed indicating direction of travel. If 
the compass is activated, aU linear movement of the viewpoint is stopped, however the 
individual can still turn or pitch the camera view. The FlyBox joystick is programmed to 
allow an individual to turn the camera view left and right up to 90 degrees and pitch the 
camera view up or down between 0 and 23 degrees to see the skyline or ground. The 
rotation and pitch of the camera view can be activated while in a travel or stationary 
mode. 

The model is displayed on three Mitsubishi Model VS-5071, 40-inch, rear- 
projection screens set in a semi-circular fashion, sixty-seven inches from the participant, 
providing the user with a 103° field of view (Figure 3.12). Participants are seated behind 
a table that supports the joystick interface. The table has enough room to hold all training 
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materials and doubles as a study station for the participant. This same area is used for in 
briefing all participants and processing paperwork for the experiment. The model display 
and workstation were separated from the lab by four-foot high wall dividers to provide 
privacy and reduce the noise firom outside sources. 




Figure 3.12. Model Display Configuration 
The triple screen configuration is used to provide the participant with the 
peripheral cues required for navigation. While maneuvering through a piece of terrain, 
an individual views his position in the context of his surroundings. Without the 
additional terrain features provided on the periphery of an individual’s vision, an 
individual is forced to delineate his position based on the 34.33° field of view rendered by 
a single screen. An example of the additional information provided to the individual is 
seen by comparing the single screen display in Figure 3.13 to the triple screen display of 
Figure 3.14. With the triple screen display, a participant is provided with traits such as 
the lone tree and trail on the left screen and the pavilion and telephone pole on the right 
screen. Moving through the environment using only one screen, the trees would have 
masked the telephone pole and pavilion to the right. If participants do not stop to look 
around at the intersection or rotate their heads during movement, they will not pick up 
these visual cues. These additional features help individuals to verify that their location 
is the trail intersection 80m to the southeast of Control Point 8 (Figure 3.15, Item A). 
Without these cues, participants may believe they are at the intersection 75m to the 
southwest of Control Point 8 (Figure 3.15, Item B), the intersection 125m south of 
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Control Point 9 (Figure 3.15, Item C), or the intersection 65m northwest of Control Point 
8 (Figure 3.15, Item D). 




Figure 3.13. Single Screen Display 




Figure 3. 14. Three Screen Display 




Figure 3.15. Map Excerpt for Three Screen Display 
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The effect of a three-screen configuration is even more apparent during 
movement. Figure 3.16 demonstrates how viewable time of a reference point increases 
as the field of view increases. During movement, individuals gauge their speed and 
position on how fast objects enter and leave their field of view. The greater the usable 
field of view the longer an object remains within view. If individuals use a reference 
point as a hand rail to keep them on course, the longer the reference point remains in 
view the more utility it provides. To demonstrate this, the graph in Figure 3.16 displays 
the curves for each field of view plotting velocity vs time. Using a generalized triangle, 
the angle a and distance a remain constant while the triangles remaining angles and sides 
h and c change as the viewpoint approaches the goal. Beginning movement at a distance 
of 212m from the goal fi equals 10°. With a center screen field of view of 30°, the 
reference point wOl drop off the center screen at a distance of 151m equals 15°). 
Expanding the field of view to 100° by using a three-screen configuration, the reference 
point would appear on the side screen from 151m to 58.5m, yS equals 50°. 




37 



Using the above distance, we can calculate the time a reference point is visible for 
each field of view by multiplying the distances by the rate of movement (Table 3.2). At 
lower velocities the difference in usable time is immense. The visible time of the 
reference point becomes more significant at greater speeds. This is due to the time 
required to identity a reference point in the environment and then locate it on the map. If 
the reference point quickly disappears from the center screen, it may not be recognized 
and utilized by the participant. Exposure on the peripheral screens may provide the 
seconds needed by the operator to use these reference points. 



Velocity (km/hr) 


3(f Field of View (sec) 


10(f Field of View (sec) 


Difference (sec) 


Ikm/hr 


222.14 


553.53 


331.39 


lOkm/hr 


22.21 


55.35 


33.14 


20km/hr 


11.11 


27.68 


16.57 


30kni/hr 


7.40 


18.45 


11.05 


40kni/hr 


5.55 


13.84 


8.29 


50km/hr 


4.44 


11.07 


6.63 


60krn/hr 


3.7 


9.23 


5.53 


70km/hr 


3.17 


7.91 


4.74 


80km/hr 


2.78 


6.92 


4.14 


90km/hr 


lAl 


6.15 


3.68 


lOOkm/hr 


111 


5.54 


3.32 



Table 3.2. Visibility Graph Time Table 

Peripheral cues used during stationary and active utilization of the model provides much 
needed information which will allow participants to disambiguate locations in the 
environment where parallel errors may occur. 

D. PARTICIPANTS 

Fifteen individuals, one female and fourteen males, served as participants in this 
experiment. The group consisted of one civilian and six Army, six Marine, and two Navy 
students from the Defense Language Institute (DLI) and the Naval Postgraduate School 
(NPS). The participants ranged in age from 28 to 39 with a mean age of 32. Participants 
had no prior knowledge of the evaluation area, nor any experience with the VE model. 
Participants were divided into one of three groups, "map only", "real-world", or "VE" 
treatment groups, based on the results of the Guilford-Zimmerman Spatial Orientation 
Aptitude Survey (GZ) (Appendix E.6). Participants received no monetary or academic 
compensation for their participation in the experiment. 
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Data collection occurred over a 71 -day period. The extended time frame for data 
collection was to allow the grass to rejuvenate from the traffic created by participants 
moving through the terrain. An additional week’s delay occurred due to an intense 
search of the area by law enforcement and military personnel looking for a missing child. 

E. RESEARCH MONITORS 

My assistant and myself acted as research monitors. Each monitor had gone 
through the course as a participant of MAJ William Banker’s thesis experiment [BANK 
97] or as a pilot participant for this experiment. Researchers followed a specified 
experimental outline (Appendix A) and a series of scripts (Appendix C) to ensure that 
each participant was presented with the same set of instructions, materials, and 
conditions. Whenever possible, participants were observed by both monitors. Research 
monitors carried additional equipment to ensure adequate supplies were on hand for the 
recording of information as well as for the safety of the participants (Appendix J). 

F. TRAIN-UP 

At the beginning of the experiment, participants were briefed on the requirements 
of the study and were asked to sign consent forms (Appendix D). After the initial brief, 
the Self Ability Evaluation (Appendix E.3), the Santa Barbara Sense-of-Direction Scale 
(Appendix E.4), and a map reading test (Appendix E.5) were administered. Next, the GZ 
test was given to measure the participants' ability to orient themselves in a 3-D 
environment. The result of the GZ test facilitated distribution of participants into the 
three treatment groups. The participants were evenly distributed based on above average 
and below average spatial orientation aptitude scores. 

The first treatment group consisted of map only participants. These participants 
studied a map of the Fort Ord orienteering course. The second, or real world group, was 
given a map of the Fort Ord orienteering course and was allowed to explore the actual 
course. The third, or VE study group, was given the map and access to the real-time 
virtual environment of the course to study. 

During the training phase of the experiment, ail reasonable attempts were made to 
replicate the procedures of Banker’s 1997 experiment. Each participant was given: a 
Participant’s Task List (Appendix G), Map Marking Instructions (Appendix H), Course 
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Clue Sheet (Figure 3.17), a laminated map of the course (Appendix F), digital photos of 
the control points (Appendix 1.2), scratch paper, pencil, and a red alcohol marker to draw 
the planned route. Virtual environment participants were given photos of the actual and 
virtual environment control points (Appendix 1.3). Real world participants were also 
given a compass for their study phase. All participants were given one hour to study the 
material provided and plan their routes. 
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Figure 3.17. Course Clue Sheet 



During the study phase, a research assistant observed each participant. Participant 
behavior was monitored and recorded on the participant’s training phase worksheet 
(Appendix M.2) for future analysis. Participants were informed when they had 30 
minutes and 10 minutes remaining. 

G. DATA COLLECTION TECHNIQUES 

Upon completion of the study phase, all study materials, except for the clue sheet, 
were collected from the participant. The participant was then taken directly to the 
starting point of the course by the shortest available route. Participants completed the 
evaluation phase via nine planned tasks, (Appendix B), and three unplanned tasks (Tasks 
3.1, 5.1, and 10). The navigation course was divided into nine unequal legs, requiring 
participants to successfully negotiate nine checkpoints. The total straight-line distance of 
the course was 2070m 

The participants established planned routes during the train-up phase with the 
participants marking their intended route on the laminated map utilizing a red marker. 
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Participants’ path knowledge was demonstrated by their ability to navigate from each 
succeeding checkpoint along their planned route. Accuracy was measured by calculating 
the number of deviations and the total distance deviated from the participant’s planned 
route. Measurements were taken using a differential global positioning system (DGPS) 
and a Newton MessagePad 130 (Figure 3.18) which were carried by the participant. 




Figure 3.18. DGPS Backpack and MessagePad 130 

The MessagePad 130 registered and stored a coordinate every five seconds as a 
participant moved and every five minutes while a participant was stationary. 
FieldWorker software was used to record the information for future analysis. Participants 
were allowed to deviate from their planned routes up to five meters while traveling on 
trails and fifteen meters during cross country movement before an error was assessed. 
This allowed the participants to explore the area and confirm their position without being 
penalized. Participants were also allowed to travel back and forth along their planned 
routes without penalty. 

A helmet camera was used as a second means of data collection. The camera was 
a Hi8 camcorder bolted to the top of a hockey helmet (Figure 3.19). A sighting apparatus 
was fabricated and affixed to the helmet to allow the camera operator to determine if the 
participant was in the field of view. The camera’s focus level was fixed at infinity and 
the camera operator stood at a distance of two or more meters from the participant to 
ensure the best possible image under these irregular conditions. The primary purpose of 
the helmet camera was to record map/con^ass checks validating the MessagePad’ s 
entries, and to provide data for behavioral analysis [BANK 97]. Participants were asked 
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to “think out load” (Appendix K) in order to provide insight to their thought process as 
they moved through the environment. 




Figure 3.19. Helmet Camera 



As a final means of data coUection, monitors manually recorded any deviation 
from the participant’s planned route and instances of map/compass checks on a black and 
white copy of the course map. This was done in the event of DGPS failure and was 
another means of verifying map/compass check locations and movement route. The 
monitors’ knowledge of the test area, their experience in creating the VE, and their 
verification of the orienteering map allowed them to reliably record participant route 
deviations and map/compass check locations. 

The experiment also examined survey knowledge by measuring (a) egocentric 
spatial knowledge using the “wheel” test and an unplanned route selection task, and (b) 
exocentric spatial knowledge using the “whiteboard” test (R. P Darken, personal 
(..ommunication, October 27, 1997). The wheel test was given to participants at 
checkpoints 2 and 4. The one-hour time limit was suspended during these tests. 
Monitors provided participants with a 12” x 12” plywood platform secured to a four-foot 
long 4x4 post (Figure 3.20). The post was fashioned to fit into a frame anchored in the 
ground near the ^ntrol point. The frame was to ensure all participants are presented with 
the platform in the same location and orientation. On top of the platform, a seven-inch 
color wheel was attached. The wheel contains 16 different colored segments, with three 
pointers fixed to the center labeled SP, 2, and 9 for tasks at CP 2 (Figure 3.21) and 
labeled 1, 6, and 8 at CP 4. Control points were chosen to ensure that only one of the 
controls had been visited by the participant prior to the wheel test and none of the 
controls were either just visited or the control to which the participant was en route. This 
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prevented participants from viewing the controls in context with their planned route, thus 
measuring only route knowledge. 




Figure 3.20. Wheel Test Platform 

The wheel was affixed to a post, which allowed the participant to move freely 
around it, while the wheel maintained the same orientation for all participants. After 
positioning the wheel table to the south of the checkpoint, the monitor instructed the 
participant to use the wheel and its three pointers to indicate the directions to the 
appropriate control points. The color wheel with no bearing marks was chosen to force 
the participant to rely solely on his survey knowledge of the environment and not to 
confuse the wheel with a compass. 

Monitors recorded actual bearings of each participant’s pointer positions using the 
color-coded segments of the wheel. More accurate measurements were taken in the lab 
using a protractor and digital photos of the participant’s color wheels. Also measured 
was the time it took the participant to complete the task and the participant’s orientation 
while positioning the arrows. Monitors recorded observations and results on the 
participant’s evaluation phase data sheet for future analysis (Appendix M.3). Upon 
completing the wheel test and before continuing on with the planned route, participants 
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were told how much time they had left to complete the course. Additional time checks 
were provided with 30 minutes and 10 minutes remaining. 



9 




Side Towards Checkpoint 



Figure 3.21. Wheel Test 

After completing the planned course, participants were asked to indicate the 
direction of CP 4, describe the best route to CP 4, and finally to navigate to CP 4 using 
the most expedient route without referencing a map or compass (Appendix L.5). 
Participants were evaluated based on the route they traveled, the number of turns that 
lead them away from CP 4, and the distance they deviated from their planned route. 
After reaching CP 4, participants were finished with the navigation portion of the 
experiment. 

Before leaving the course, one final unplanned test was administered. The 
whiteboard test measures a participant’s exocentric survey knowledge of the 
environment. For this test, monitors provided participants with a white magnet board and 
ten magnets corresponding to the start point and the nine respective checkpoints. 
Researchers then instructed participants to place the magnets on the board in proper 
relation to each other, as the points would appear from a top-down view of the terrain 
(Figure 3.22). A digital photo was taken of the participant’s magnet layout and later 
analyzed to measure the accuracy of the representation’s relative bearings between 
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checkpoints and the relative distances of each leg of the course. An overall score is 
computed based on the total angular deviation compared to calculations derived from the 
orienteering map (Chapter IV, Section A.5.b). 




, Figure 3.22. Whiteboard Test 

After completing the whiteboard test, participants were taken back to NFS for the 
debriefing. During the debriefing, participants were asked to complete a questionnaire 
about their method of training and the course (Appendix A.7.f). Participants were also 
walked through the route they followed which was plotted on an aerial photograph of the 
course. The data exhibited on the photograph was exported from the MessagePad, 
converted into a format readable by ArcView software, and displayed on a 21” monitor. 
As they were taken through their actual route, participants were questioned on why they 
felt they had deviated from their planned routes. 

During the execution of the course, participants were only required to wear the 
portable GPS backpack and carry their clue sheet. All other supplies, to include drinking 
water, were carried by the research monitors. 
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IV. ANALYSIS 



A. RESULTS 

1. General Information 

The experiment is designed to test a primary and secondary hypothesis 
concerning the navigational knowledge of participants exposed to different training 
methods. To determine overall navigational performance, participants were evaluated on 
their route and survey knowledge of the environment whUe conducting an orienteering 
course through the target teirain. 

a. Primary Hypothesis: 

Given an hour exposure to training materials, participants with access to a real- 
time virtual environment will outperform those who are exposed to only a map and 
photos of the control points for the same time duration. 

b. Secondary Hypothesis: 

Given an hour exposure to training materials, participants with access to the real 
world will outperform those who are exposed to a real-time virtual environment of the 
same time duration. 

c. Sub Hypotheses: 

1) Route Knowledge 

a) Real world participants will commit fewer errors per control point 
attempted than VE participants. Virtual environment participants 
will commit fewer errors per control point attempted than map 
only participants. 

b) Real world participants will travel less distance per error before 
discovering and correcting their errors than VE participants. 
Virtual environment participants will travel less distance per error 
before discovering and correcting their errors than map only 
participants. 

c) Real world participants will perform fewer map and/or compass 
checks per control point attempted than VE participants. Virtual 
environment participants will perform fewer map and/or compass 
checks per control point attempted than map only participants. 
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2) Survey Knowledge 

a) Real world participants will have a smaller average delta angle on 
the wheel test than VE participants. Virtual environment 
participants will have a smaller average delta angle on the wheel 
test than map only participants. 

b) Real world participants will have a smaller average delta angle on 
the whiteboard test than VE participants. Virtual environment 
participants will have a smaller average delta angle on the 
whiteboard test than map only participants. 

c) Real world participants will have fewer errors during the execution 
of the unplanned route from CP9 to CP4 than VE participants. 
Virtual environment participants will have fewer errors during the 
execution of the unplanned route from CP9 to CP4 than map only 
participants. 

2. Power Analysis 

The tests conducted are two-way analysis of variances (ANOVAs) of study group 
and spatial aptitude (Guilford-Zimmerman). The sample size is fifteen participants. An 
□ of 0.05 was used, resulting in a power value (!-□) of 0.1095. As a result, the ability to 
detect alternative hypotheses is poor. This suggests that drawing any conclusions based 
exclusively on a failure to identify a positive effect on any factor is imprudent. A 
multiple analysis of variance (MANOVA) was not performed since each measure was 
analyzed to determine statistical significance for only one sub hypothesis. Simultaneity 
of effects was not considered critical. 

In general, most of the graphs presented here are box plots on primary factors 
indicating the mean, standard deviation, and standard error. In addition, some graphs 
depict extreme data points as dots. 

3. Normalization of Data 

Many of the measurements used for analysis in this experiment occur over time 
and distance. Some participants were not given certain tests because they were unable to 
reach test locations prior to the one-hour time limit. To make participant data 
comparable, several measurements were normalized over the number of controls 
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attempted. This placed each participant’s data in a rational format for correlative 
analysis. 



4. Route Knowledge 

Route knowledge is assessed by analyzing the errors committed, the average 
distance per error, and the average number of checks performed by each participant. 
a. Errors 

Five distinctive navigation errors were observed and recorded during the 
execution of the experiment. These errors were weighted and treated equally in the 
analysis of this data. 

A parallel error occurs when participants mistake one piece of terrain for another 
or travel a parallel path more than 5m off their planned route when traveling along roads 
or trails and 15m off their planned route when traveling cross-country. An example of a 
parallel error is when an individual is located at an intersection west of his perceived 
location (Figure 4.1). 



- Actual Location 

- PCTceived Location 

Figure 4.1. Parallel Error 

Mirror errors occur when participants reach a decision point and mistakenly 
choose to travel the route that takes them in the mirror opposite direction of the correct 
path. 





- Original Direction of Travel 

- Correct Route 

- Mirror Ettot 



Figure 4.2. Mirror Error 
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Figure 4.2 is an example of a mirror error would be, if a participant comes to a fork in the 
road and mistakenly chooses the left trail when the right trail was the route originally 
intended. 

Participants are assessed an out-of-bounds error when they travel outside the 
roads that enclose the course. The participants are not informed of their location or the 
proximity of other boundaries. The participants are only told that they have left the 
course area and must return to the other side of the boundary road. No additional 
distance error is recorded. 

A reorientation error occurs if participants have been off their designated route 
for fifteen continuous minutes and are not making progress towards their intended control 
point. The participants are stopped, shown their location on the map, reoriented to the 
ground, and given sixty seconds to mark their new route to the intended control point. 
The distance fi’om this location until the participant locates the correct control point is 
added to the parallel or mirror error which was at the origin of the participant’s 
disorientation. 

A compound error is assessed if participants commit a parallel or mirror error on 
the newly plarmed route resulting from an assessed reorientation error: The distance is 
measured and recorded for this error fi'om the location where participants are more than 
5m off their planned route when traveling along roads or trails and 15m off their planned 
route when traveling cross-country until they reacquire the newly planned route. 

Parallel, mirror, and compound errors are weighted equaUy. Due to information 
being passed to the participant during the assessment of out-of-bounds and reorientation 
errors, they can be thought of as umequested map checks. These errors are used in the 
calculations of the Map Check Scores (Chapter IV, Section A.4.c) instead of being used 
in determining the participants’ error scores. This prevents the errors from having twice 
the impact on the participants’ overall scores and is more representative of the event. For 
each leg of the course, each participant’s errors were recorded and summed into a Leg 
Error Score. A participant’s Normalized Average Error Score is the summation of the 
individual Leg Error Scores divided by the number of controls attempted (Appendix 
0.3.a). 
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Figure 4.3 shows Normalized Average Error Scores of all three treatments with a 
lower score indicating better performance. The map and real world conditions are 
approximately the same at 0.7 errors per attempt. The VE group lags behind with a group 
mean of 1.1. Also note the real world participants had a far greater variance indicating 
that while some participants had a higher performance others had exceptionally poor 
performance. 

Box Plot 

Grouping Variable: Group 
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Figure 4.3. Interaction Box Plot for Error Per Controls Attempted (Group) 

The means between groups are not statistically different, F(2,12) = 1.053, P = .3789. 
Direct observations suggest -that map participants outperformed VE and real world 
participants by committing fewer errors per control attempted. Map participants 
appeared to follow their planned routes better indicating they had better route knowledge 
of the course. Real world and VE participants (VEl, VE3, RW3, and RW5) who made 
better progress during the training phase committed fewer errors per control attempt than 
those participants who made little progress during the training phase. This suggests that 
individuals who make it further in the environment during their exposure to a VE or the 
real world demonstrate better route knowledge by committing fewer errors. This also 
suggests that difficulties encountered during the training phase carried over to the 
execution phase of the experiment (Chapter FV, Section B.9). 
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b. Distance Traveled per Error 

Determining how long it takes the participant to recognize and recover from the 
error is as important as identifying when a participant makes an error. To determine this, 
the distance the participant travels is measured from the point an error is committed until 
the participant returns to the planned route. These measurements are determined by 
comparing the participant’s planned route to the data collected from the Global 
Positioning System. Both pieces of information are loaded into and displayed in 
ArcView on the aerial photo of the course. Using the ArcView measuring tool, 
measurements are then taken of any differences based on the route error criteria. The 
experiment parameters specified that measurements be taken from the location where 
participants are 5 or 15m respectively from their planned route until they reacquire their 
planned route was measured and recorded for this error. The measurements are summed 
and divided by the total number of errors to determine the average distance per error. 
These figures are computed for individual legs and the complete course for each 
participant resulting in the Leg Distance Per Error Score and the Distance Per Error Score 
(Appendix 0.3. a). Dividing the average distance per error by the number of controls 
attempted results in the Normalized Average Distance Per Error. 
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Figure 4.4. Interaction Box Plot for Normalized Average Distance Per Error (Group) 
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The Normalized Average Distance Per Errors of all three treatments with a lower 
score indicating better performance are shown in Figure 4.4. The map and real world 
conditions are approximately the same at 80m per error per attenpt. The VE group lags 
behind with a group mean of 131m. 

The means between groups showed no statistical difference, F(2,12) = .479, 
P = .6305. Direct observations indicate that real world and map participants 
outperformed VE participants by traveling less distance per error per control attempted. 
This implies that map and real world participants were equally adept at identifying errors 
and correcting them. Note that the VE participants had a far greater variance indicating 
that while some participants had a higher performance, one individual had exceptionally 
poor performance. Virtual Environment Participant #2 was an extreme outlier for 
distance measurements since the participant traveled over 3000m on his first error 
encircling the entire course (Appendix N, Figure N.63). After removing this participant 
from the analysis, results for aU three groups are roughly identical, (F(2,ll) = .003, 
P = .9965). This suggests that there is, in fact, no difference between groups when it 
comes to identifying that an error has been committed and being able to recover from an 
error (Figure 4.5). 
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Figure 4:5. Modified Interaction Box Plot for Normalized Average Distance Per Error 
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c. Map and Compass Checks 

Participants were allowed to request three distinct types of navigational checks 
while on the evaluation portion of the exercise. Each check was timed and recorded by 
research personnel on a black and white copy of the course map. Participants were 
allowed to request as many checks as they felt necessary for them to conduct the course. 
Participants were allowed to request consecutive checks if they need them. During a map 
check, participants were allowed to view the laminated map marked with their designated 
course for thirty seconds. For a compass check, participants were given an orienteering 
compass for thirty seconds. For a map and compass check, participants were given the 
laminated map marked with their designated course and an orienteering compass for sixty 
seconds. When not being used for a map and/or compass check, the research monitor 
maintained all materials. 

Map and con^ass checks are weighted at 1.0. A combination map and compass 
check is weighted at 1.5 for the additional information that can be gathered utilizing the 
two in tandem. For a change of route, a participant is assessed a .5 weight against a map 
check. The penalty is assessed due to the information which can be gained while plotting 
the new route on the map, however a full penalty is not assessed since participants must 
have knowledge of their location and the environment if they wish to change their route. 
If participants misidentify their location and therefore plot a bad route from their current 
position, a parallel error is assessed as participants initiate movement from their current 
position. An out-of-bounds error is weighted at 2.0 because participants receive 
additional information from the research monitor who tells participants they have left the 
course boundaries. An out-of-bounds error also indicates that participants have lack of 
knowledge of their location with respect to the course. The reorientation error is 
weighted at 3.0 due to assistance provided by the research monitor who informs 
participants they are off their designated route and shows them exactly where they are on 
the map before the participants plot a new route to the intended control point. 

Map Check Scores were calculated for each leg of the course and a Total Map 
Check Score was calculated by adding the individual leg scores. Dividing the Total Map 
Check Score by the number of controls attempted normalizes the value. This value is 
known as the Normalized Map Check Score (Appendix 0.3. a). Figure 4.6 displays the 
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Box Plot 

Grouping Variable: Group 




Figure 4.6. Interaction Box Plot for Normalized Map Check Score (Group) 
Normalized Map Check Scores of all three treatments with a lower score indicating better 
performance. The map condition outperformed the other two with a mean of roughly 1.5 
checks per control attempt. The VE and real world conditions were relatively the same at 
approximately 3 checks. The means between groups are not statistically different, 
F(2,12) = .838, P = .4564. Direct observation indicates that the map only group 
participants performed less map and compass checks than VE and real world participants. 
This suggests that map only participants had more confidence in their memory and 
mental maps than the VE and real world participants who required more checks per 
control attempted to resolve differences in their mental maps and the actual environment. 
This is due to map participants concentrating solely on the map during the training phase 
which resulted in a better mental facsimile of the map. 

5. Survey Knowledge 
a. Wheel Test 

Participants were tested twice using the wheel test (Chapter 3.E), once at Control 
Point 2 and again at Control Point 4. Digital photos of the participants’ answers were 
con^ared to images of the correct answer. Measurements were taken of the angle 
variances. The participant’s absolute values of the angle deviations are summed for each 
control point. Each participant’s total angle deviation for Control Point 2 and Control 
Point 4 wheel test sites are used to determine the participant’s egocentric survey 
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knowledge. The total angular differences are stored in Average Angular Difference CP2 
and in Average Angular Difference CP4 receptively (Appendix 0.4.b). Four participants 
were not administered the wheel test at CP4 since they failed to locate the control point. 
To compare scores across participants, Average Wheel Test Angular Variance scores are 
normalized by adding the absolute values of the angular differences and dividing the sum 
by the number of control points the participant identified, 3 or 6 control points. 

The Average Wheel Test Angular Variances of all three conditions with a smaller 
deviation indicating better performance are shown in Figure 4.7. All three conditions are 
approximately the same at 26.5° per error per attempt. The means between groups 
showed no statistical difference, F(2,12) = .012, P = .9879. Direct observation suggests 
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Figure 4.7. Interaction Box Plot for Average Wheel Test Angular Variance (Group) 
that map and VE participants outperformed the real world group who had a larger 
standard deviation. This deviation indicates a greater variance amongst participants 
within this group with most participants performing well and some participants 
performing poorly. Without this variance, performance by real world participants may 
have shown better results. The two real world participants who had the greatest average 
angle variance (RWl and RW4), took longer to perform the wheel test and found fewer 
control points than the other real world participants (Appendbc O). This indicates they 
had less route and survey knowledge than their fellow real world participants. 
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b. Whiteboard Test 

Each participant’s Whiteboard (Chapter 3.E) results were analyzed to determine a 
participants exocentric survey knowledge. This was acconqjlished by calculating the 
angle differences between each control point. The angular variances were determined 
using a metric intended to normalize results for simple comparisons across participants. 
The technique begins with capturing results by taking a top down digital photo of each 
participant’s Whiteboard. The image is down loaded to a PC and imported into Adobe 
Photoshop. Once loaded into PhotoShop, the image imperfections are removed and the 
photo is squared. Using the PhotoShop navigator tool, “x” and “y” coordinates are taken 
for the center of each control point. The coordinates are fed into an Excel spreadsheet 
that calculates distance between successive control points (SP, 1, 2, 3, 4, 5, 6, 7, 8, 9, SP). 
Dividing each leg measurement by the sum total of all distances for each participant 
normalized the distances. The resulting normalized distance measurements are used in 
conjunction with the Pythagorean Theorem to calculate the angles between the successive 
control points. The resulting angles are compared to the actual angles between control 
points. The actual angles are calculated from the “x” and “y” coordinates from a 
digitized course map. The resulting variances and their absolute sums are provided along 
with participant Whiteboard coordinates and measurements in Appendix 0.5. Sanitized 
participant Whiteboard images are provided in Appendix N. 
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Figure 4.8. Interaction Box Plot for Average Whiteboard Angular Variance (Group) 
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Figure 4.8 displays the Average Whiteboard Angular Variances of all three 
treatments with a lower score indicating better performance. The means between groups 
are not statistically different, F(2,12) = 1.056, P = .3781. Direct observations indicate that 
on average, map and VE conditions had smaller average delta angle, approximately 21.5°, 
than the real world group, at roughly 27.5°. This indicates that the map and VE 
participants had a better exocentric reference than real world participants. The enhanced 
level of performance by the map group is due to continued exposure to the map. The VE 
participants’ enhanced performance results from their exposure to the top down view 
with a “you are here” arrow. Because of these two aspects of the training phase, neither 
group had to worry . at becoming lost in the environment. Real world participants had 
to rely on their navig„ .on skills during the training phase to ensure they did not become 
disoriented. The effort required by real world participants to ensure they did not become 
lost reduced the time available to them to study the map and terrain. This resulted in their 
reduced level of excocentric knowledge of the environment. 
c. Unplanned Route Execution 

The unplanned route test was administered after the successful conq>letion of the 
planned navigational task. The task consists of explaining the route the participant would 
take to reach Control Point 4 from Control Point 9 without referencing a map. The 
participant is then required to navigate to Control Point 4 from Control Point 9 without 
the use of a map or compass. Six of the participants completed their planned course and 
were administered the Unplanned Route Task. Of the six participants administered the 
task, only one navigated to Control Point 4 by a means other than the route used to reach 
'^ontrol Point 9 from Control Point 8. This was VE Participant Number 1 (Appendix 
J.6). 

None of the participants tested committed an error enroute from Control Point 9 
to Control Point 4. The average time to complete this task was five minutes and twenty- 
eight seconds. The data from the Unplanned Route Task is provided in Appendix 0.6. 
For the participants who were administered the test, their performance indicates 
tendencies to travel previously visited terrain in order to link formerly unassociated 
controls. This supports the belief that landmarks are grouped and associated by common 
links even though they have never previously been closely coupled. Since only 40% of 
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the participants were administered this test, no conclusions can be draw between the 
effects of training tools on the performance of this task. 

In conjunction with the Wheel Test and Whiteboard results, these figures indicate 
that the survey knowledge amongst all participants was not affected by the method of 
study. This could be a result of aU participants having access to the same map from 
which to gain their survey knowledge. This coincides with Thomdyke’s research [THOR 
82 ] which showed that survey knowledge is gained through the study of external sources 
such as maps. 

6. Navigational Performance by Training Condition 

The results of the analysis indicate no statistical significance based on training 
condition. In general, direct observations suggest that map participants outperformed VE 
and real world participants. This can be the result of numerous factors including map 
fidelity, spatial ability, and route complexity. 

A direct comparison of results is paradoxical because of the varying route 
conplexities. In this experimental paradigm, participant performance on short difficult 
routes was compared to performance on longer easier routes. Problems encountered with 
identifying meaningful performance measurements, recording data, and conducting 
appropriate analysis were magnified since no two routes were identical. Providing each 
participant with a pre-planned route could have alleviated many issues. The result would 
have been an experiment testing the effect of the training conditions on route knowledge. 
Although interesting, a more important issue is how survey knowledge is affected by the 
different conditions. If VEs are to be a valuable navigational tool for the military, they 
must provide survey knowledge of the environment. Survey knowledge of an 
environment allows individuals to vary their routes based on current conditions and 
allows for more efficient movement through the environment. This is why participants 
must be allowed to explore the environment and plan their own routes. 

B. DISCUSSION 

1. Landmark Knowledge 

To determine the participants’ level of landmark knowledge, they were evaluated 
on their ability to locate and identify the control points. While executing the course, 
participants have three possible results on each leg: control point found, control point not 
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found, and time expired in route. Control point found indicates that the participant 
located and touched the appropriate control point. Participants receive lull credit for 
locating the appropriate control point. An unfound control point is defined as a control 
point that participants could not locate because they were off their planned route and time 
had expired. A participant receives 33.33% credit towards a control found for attempting 
to locate the control point. If time expires while a participant is in route to the next 
control point, the participant is given 66.66% credit towards a control found for 
attempting to locate the control point. Participants are awarded this credit if they are 
enroute to the control point and on their preplanned route. The sum of these values is a 
participant’s Landmark Knowledge Score (Appendix 0.3). 

Figure 4.9 displays the Landmark Knowledge Scores of all three conditions with a 
higher score indicating better performance. The means between groups are not 
statistically different, F(2,12) = .563, P = .5840 (Figure 4.1). Direct observation suggests 
that on average, map and real world participants located more controls than VE 
participants. This cursory analysis of the data does not indicate whether the participants 
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Figure 4.9. Interaction Box Plot for Landmark Knowledge Score 
were having the majority of their problems during coarse movement through the 
environment (Appendix Q) or while searching for the control in the general location of 
the objective. A better measurement of landmark knowledge would have been to 
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measure the time or distance traversed as a participant searched within the general area of 
the control (25m). 

2. Route Complexity by Training Condition 

Figure 4. 10 displays the ISOM Average Planned Route Conplexities Scores of all 
three treatments with a lower score indicating an easier route. The VE condition planned 
less aggressive routes than the other two with a mean route complexity of roughly 1.5. 
The map condition followed closely with a mean route complexity of approximately 1.75 
and real world conditions trailed with 2.0. The means between groups are not statistically 
different, F(2,12) = 1.247, P = .3222. Direct observations indicate that on average. 
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Figure 4. 10. Interaction Box Plot for ISOM Average Planned Route Complexity (Group) 
participants in the VE group tended to plan less con^lex routes. This indicates that the 
VE provided participants with information concerning the conplexity of the environment 
which they could not gain from the map or failed to gain from the real world. Time 
compressed training allowed VE participants to explore more of the terrain than the real 
world participants were able to traverse during the training phase. This afforded VE 
participants the opportunity to plan less complex routes by taking advantage of the 
information gained from the VE. Map participants did not have the opportunity to 
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translate their propositional knowledge gained of the environment into imagery. This 
may have prevented them from identify more simplistic routes through the course. 

3. Wheel Test Results Visited vs Unvisited Control Points 
a. Visited vs Unvisited Control Points 

A post-hoc analysis was conducted on the average angular differences between 
previously visited and unvisited control points on the Wheel Test. The analysis was done 
to see if participants had better conceptual placement of controls visited vs controls not 
yet visited indicating a better egocentric conception of visited controls. 




Figure 4.1 1. Scattergram for Wheel Tests Results by Controls Visited and Unvisited 



A line with a slope of one would indicate similar angular difference between 
controls visited and controls not visited. Horizontal regression lines suggest that 
performance on visited controls was better than those for unvisited controls. Figure 4. 1 1 
exhibits nearly horizontal regression lines for aU three groups. This indicates that 
participants had better spatial placement of visited controls than unvisited controls. Map 
participants showed scatted results suggesting that placement of controls in each 
individual’s mental representation of the world varied based on how they grouped the 
controls. Further research is needed to verify these suggested results and to determine if 
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objects further along on the course are more difficult to place within the mental 
representation than objects closer on the route. 

b. Control Point 2 vs Control Point 4 



Additional post-hoc analysis was performed to determine if participant 
performance improved or degraded on the Wheel Test conducted at CP4 compared to the 
test at CP2. The comparison is meant to determine if individual performance improves 
the longer that the participant is exposed to the actual environment. The inference is that 
increased exposure time allows individuals to resolve the differences between their 



mental maps and the actual terrain. 
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Ave Angular Diff CP2 = 151.159 - 5.317 * Ave Angiular Diff CP4 ; R''2 = .518 (.Map) 
Ave Angular Diff CP2 = 19.652 - .247 * Ave Angiular Diff CP4 ; R^2 = .621 (.Virtual Env) 
Ave Angular Diff CP2 = 6.46 + .689 * Ave Angiular Diff CP4 ; R^2 = .43 (Real World) 



Figure 4.12. Scattergram for Wheel Tests Results by Test Location 
Only participants who performed the Wheel Test at both locations were used in 
this comparison. Although no conclusions can be drawn from the results, observations of 
the research personnel suggest that on average. Figure 4.12 shows assorted results 
depending on the group. Map participants had mixed results for average wheel test 
scores at Control Point 2. They showed a clustering of results around the 25° variance 
measurement for their scores at Control Point 4. This produces in a nearly vertical 
regression line indicating improved and standardized results at Control Point 4 with 
assorted results at Control Point 2. 



63 



The VE group results produce an approximate horizontal regression line. This 
denotes a clustering of results around the 15° mark for their scores at Control Point 2 
while exhibiting more dispersed results At Control Point 4. This suggests di minishin g 
performance for the VE participants between the two test sites. Real world participant 
data resulted in a positively sloped regression line that suggests that the average 
performance of individuals remained relatively constant between the two testing sites. 
Overall results suggest that increased exposure to the environment had little effect on 
performance of participants egocentric representation of the environment. 

4. Spatial Ability Post-Hoc Correlation 

To assist in determining additional tests that may predict individual navigational 
performance, individual post-hoc ANOVAs were run with the independent variables 
being the Self Evaluation Bar Test, the Santa Barbara Sense-of-Direction Scale, the Map 
Reading Test, and the GZ Test. The dependent variables were the same as those used for 
analyzing navigational performance (Chapter 4.A.5). No significance was shown when 
running the dependent variables against the independent variables Self Evaluation Bar 
Test, Santa Barbara Sense-of-Direction Scale, and Map Reading Test. Potential 
statistical significance exists with regards to the independent variable GZ Ability Groups. 
Only the analysis of the GZ Ability Group results will be discussed. All dependent 
variables analyzed use the same criteria utilized in the factorial analysis of the dependent 
variables by the independent variable group (Chapter IV, Sections A.4 &. A.5). 
a. Route Knowledge 
1) Errors 

Figure 4.13 displays the Errors Per Control Attempted based on the two Guilford- 
Zimmerman conditions with a lower score indicating better performance. The results do 
not display statistical significance between the two groups, F(l,12) = 4.040, P = .0656. 
Direct observations indicate that on average, participants who scored higher on the 
GuUford-Zimmerman Test committed fewer errors than those who scored lower. This 
suggests that individuals with higher GZ scores are better able to follow their planned 
routes. This could be due to route selection, organization of mental map, or memory 
skills. Further research is needed to determine what factors the GZ score influences and 
what impact these variables have on navigational ability. 
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Box Plot 

Groupina Variable: GZ Abilitv Group 




Figure 4.13. Interaction Box Plot for Errors Per Control Attempted (GZ) 

2) Distance Traveled Per Error 

The Normalized Average Distance Per Error of the two conditions is shown in 
Figure 4. 14 with lower score indicating better performance. The groups show statistical 
significance, F(143) = 9.702, P = .0082. Participants who scored higher on the GuUford- 
Zimmerman Test traveled shorter distances per error. This implies that individuals with 
higher GZ scores identify their errors and can recover from them faster than those 
individuals who have lower GZ scores. Since participants with higher GZ scores 




Figure 4.14. Interaction Box Plot for Normalized Average Distance Per Error (GZ) 
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committed fewer errors and recovered from them faster than participants with lower GZ 
scores, they were also able to attempt more control points. This helped to improve their 
self-confidence as they continued on the course and reduce their scores that were 
normalized by controls attempted. This provides credence to and justification for the 
observation that they were able to obtain better route knowledge of the environment than 
the average participant with a lower GZ score. 

3) Map and Compass Checks 

Figure 4.15 displays the Normalized Map Scores based on the two Guilford- 
Zimmerman conditions with a lower score indicating better performance. The results 
failed to show statistical significance between the two groups, F(l,13) = 4.254, P = .0597. 




Figure 4.15. Interaction Box Plot for Normalized Map Score (GZ) 

Although no conclusions can be drawn from the results, direct observation suggest that 
on average, participants who scored higher on the GuUford-Zimmerman Test required 
fewer checks and corrections by monitors than those who had lower GZ scores. This 
implies that individuals with higher GZ scores had more confidence in their memory and 
mental maps than individuals with lower GZ scores. This is a result of their spatial 
ability which allowed them to organize a specific set of navigation cues or develop an 
accurate mental map. These skills also allowed them to confirm or modify their mental 
representations as they were presented with conflicting information during execution of 
the course (Appendix Q). 
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b. Survey Knowledge 
1) Wheel Test 

The Normalized Average Distance Per Errors of the two conditions with a smaller 
deviation indicating better performance is shown in Figure 4.16. The means between 
groups shows a statistical difference, F(l,13) = 6.064, P = .0285. Direct observations of 
the research personnel suggest that on average, participants who scored higher on the 
Guilford-Zimmerman Test had smaller average delta angles indicating they had a better 
exocentric reference than lower GZ score participants. This is a result of their enhanced 
ability to fix their position in their mental representations and then rotate their mental 
maps to identify the relative position of the control points based on their location. 
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Figure 4. 16. Interaction Box Plot for Average Wheel Test Angular Variance (GZ) 

2) Whiteboard Test 

Figure 4.17 displays the Average Whiteboard Angular Variances based on the two 
Guilford-Zimmerman conditions with a smaller deviation indicating better performance. 
The results failed to show statistical difference between the two groups, 
F(1,13) = .128, P = .7258. Direct observations of the research personnel suggest that on 
average, the Guilford-Zimmerman Test results had no impact on the average delta angles 
indicating both higher and lower scoring individuals have similar abilities to acquire valid 
exocentric representations of the environment. Since this test did not require participants 
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Figure 4.17. Interaction Box Plot for Average Whiteboard Angular Variance (GZ) 
to conduct a mental rotation of the environment, spatial ability did not have the same 
affect on performance as it did on the wheel test (Chapter IV, Section B.4.b.l). This 
indicates that excocentric spatial knowledge is independent of egocentric spatial 
knowledge with regards to spatial ability. As a result, survey knowledge must be 
evaluated based on both excocentric and egocentric measures. 

3) Unplanned Route Execution 

The Normalized Average Distance Per Errors of the two conditions is shown in 
Figure 4.18 with lower score indicating better performance. The means between groups 
indicates statistical difference, F(l,13) = 7.316, P = .0180. Direct observation suggests 
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Figure 4.18. Interaction Bar Plot for Unplanned Route Execution (GZ) 
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that on average, a participant who scored higher on the Guilford-Zimmerman Test was 
more likely to complete the course in time and be administered the unplanned route task. 
Since participants with higher GZ scores were more hkely to complete the course, it is 
more likely they had acquired the survey knowledge required to perform the unplanned 
route task than those individuals who had lower GZ scores. This implies that individuals 
with higher GZ scores have the ability to obtain overall route and survey knowledge 
faster than those individuals who have lower GZ scores. 

c. Route Complexity by Spatial Ability 
1) Route Planning 

Figure 4.14 displays the ISOM Average Planned Route Con^lexities based, on the 
two Guilford-Zimmerman conditions with a lower score indicating easier route. The 
results suggest statistical significance between the two groups, F(l,13) = 8.614, 
P = .01 16. Direct observations indicate that on average, participants who scored higher 
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Figure 4.19. Interaction Box Plot for ISOM Average Planned Route Complexity (GZ) 
on the Guilford-Zimmerman Test planned less difficult routes (1.417, high beginner) than 
those who scored lower (2.422, high intermediate). This imphes that those individuals 
with higher GZ scores are better able to identify simple routes through this environment. 

The ability of higher GZ score participants to identify and plan more simplistic 
routes is the result of their ability to conduct mental rotations of the map symbols or 
environmental imagery. This allowed them to mentally visualize decision points 
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throughout the environment and determine which features would provide the best 
directional, identification, and reassurance signs (Appendix Q). These signs helped them 
to confirm their position and orientation along their planned route prior to executing the 
course. 

2) Route Planning vs Average Error Score 
Figure 4.20 displays the ISOM Average Planned Route Conplexities based on the 
two Guilford-Zimmerman conditions and Normalized Average Distance Per Error with a 
score in the lowe eft comer indicating easier route and better performance. The results 
do not indicate statistical significance between the two groups, F(2,14) = 3.710, 
P = .0557. Direct observations indicate that participants who scored higher on the 
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Figure 4.20. Scattergram for ISOM Average Planned Route Conplexities (GZ) 



Guilford-Zimmerman Test planned less difficult routes and were able to follow those 
routes better than those who scored lower. This implies that individuals with higher GZ 
scores are better able to plan routes that allow them to identify when they have left their 
routes. This allows them to quickly recover fi’om their errors and continue on the planned 



route. 



d. Navigational Performance by Spatial Ability 

The results of the analysis of route and survey knowledge based on spatial ability 
suggest possible statistical significance. This implies that spatial aptitude has a more 
profound impact on navigational performance than training effect. This could be due to 
the individuals’ ability to understand the con^lexity of the task, plan a more appropriate 
route, and resolve differences in their mental maps while executing their planned routes. 
The data does not imply that lower scoring GZ participants could not execute their 
planned route. Instead it suggests that they did not understand the complexity of the task 
and planned routes which were too complex to execute within the allotted time. 

5. Debriefing Questionnaire 

To provide a qualitative analysis of the tools and course used for the experiment, 
participants were given a debriefing questionnaire. The map and real world groups’ 
version of the questionnaire (Appendix E.8) did not have any questions concerning the 
experiment model or its interface. The VE group questionnaire asked specific questions 
regarding the experimental model and interface. A five-point scale (1-5) was used for the 
questionnaire. 

a. Map Questions 

Qualitative analysis of the map indicates that map participants had more 
confidence in the 1:5,000 orienteering map than their real world or VE counterparts 
(Questions MapQ2 - MapQ6). This is partially due to the need for real world and VE 
participants to resolve differences in the mental maps they created during the training 
phase and the course they were running for the execution phase (Chapter IV, Section 
B.ll). Real world and VE participants showed less confidence in the map’s ability to 
depict vegetation even though the experiment map used the more descriptive depictions 
of orienteering maps than the traditional military representations. The. mean qualitative 
score was 3.4, adequate representation, and scores ranged from 1 to 5. The participants 
felt the map allowed them to easily plan their routes (Question MapQ7). This indicates 
that the map was a useful tool for all participants. 
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Figure 4.21. Cell Bar Chart for Debriefing Questionnaire (Map Questions) 



Code 


Question 


MapQl 


Was the map easy to read? 


MapQ2 


Was the map easy to undCTstand? 


MapQ3 


Were the trails & roads adequately shown on the map? 


MapQ4 


Were the man-made structures adequately shown on the map? 


MapQ5 


Were the obstacles adequately shown on the map? 


MapQ6 


Was the vegetation adequately shown on the map? 


MapQ7 


Using the map, how difficult was it to plan your route? 



T able 4.1. Map Questions 



b. Course Questions 

Qualitative analysis of the course indicates that map participants felt the course 
was moderately difficult and that control points were well marked and located in the 
general location they had expected them (Questions CourseQl - CourseQ3). On average, 
participants felt that trails had been trampled down between controls with scores ranged 
from 1-4 (Question CourseQ4). This did not help participants locate CP2 or CP4. 
Trails only provided confidence when participants were on their desired route. The trails 
created confusion when participants were disoriented. The reported difficulty of 
remembering one’s planned route ranged from easy to hard with a mean value of 3.667 
indicating that the average participant found this to be moderately difficult (Question 
CourseQS). Virtual environment participants felt they were better able to remember their 
planned routes than did the map or real world participants. 
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Figure 4.22. Cell Bar Chart for Debriefing Questionnaire (Course Questions) 



Code 


Question 


CourseQl 


How difficult was the course? 


CourseQ2 


Were the control points well marked? 


CourseQS 


Were the control points located where you expected them? 


CourseQ4 


Had routes been trampled down leading to the control points? 


CourseQS 


Did you have difficulties remembering your planned route? 



Table 4.2. Course Questions 



c. Miscellaneous Questions 

Map participants felt they had sufficient time to plan and study their routes 
(Questions MiscellaneousQ2 and MiscellaneousQS). Virtual environment and real world 
participants felt they could have used more time. Resolving differences in the 
environment requires time and exposure to the terrain (Chapter IV, Section B.ll). The 
reasons for this are similar to the rationale behind why participants in the VE and real 
world groups were less confidant in the map (Chapter FV, Section B.4.a). The time used 
to resolve these issues distracts from the time allotted to plan and practice a route through 
the environment. 
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Figure 4.23. Cell Bar Chart for Debriefing Questionnaire (Miscellaneous Questions) 



Code 


Question 


MiscQl 


Did you en joy this experiment? 


MiscQ2 


Did you feel the training phase was long enough? 


MiscQS 


Did you feel the training phase was too short? 


MiscQ4 


Do you feel the training familiarized you learn the environment? 


MiscQS 


Did you feel confident in navigating the terrain without a map or compass? 



Table 4.3. Miscellaneous Questions 



Map participants felt more confident in navigating through the environment 
without the use of the map (Question MiscellaneousQS). Real world participants felt less 
confident than their map and VE counterparts. This is due to the real world participants’ 
inability to explore the entire environment during the training phase. Four of the five real 
world participants faded to make it past Control Point 6 during the study phase. These 
participants were unsure of the environment on the south side of the course. The 
southern half of the course was depicted as having more undergrowth and greater 
changes in elevation than the northern half of the course (Appendix F.6). This limited 
exposure to the environment translated into a lack of confidence in navigating without the 
use of the map. 

d. Model Questions 

Map participants felt the model correlated well with the map and was easily 
viewable (Questions ModelQ2 and ModelQl). Participants felt that the elevation 
representation and man-made structures were well represented in the model (Questions 
ModelQ? and ModelQ4). These elements of the model assisted participants with 
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identifying the general area in which they would locate the controls and enhanced their 
confidence in the training they had received (Questions ModelQ9 and ModelQl 1). 
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Figure 4.24. Cell Bar Chart for Debriefing Questionnaire (Model Questions) 



Code 


Question 


ModelQl 


Was the model clear and viewable? 


ModelQ2 


Did the model coincide with the map? 


ModelQS 


Were the trails & roads adequately represented in the model? 


ModelQ4 


Were the man-made structures adequately represented in the model? 


ModelQS 


Were the obstacles adequately represented in the model? 


ModelQ6 


Was the vegetation adequately represented in the model? 


ModelQ? 


Were changes in elevation adequately represented in the model? 


ModelQS 


Did the model help you identify the control points within the last 50m? 


Model09 


Did the model help you identify the general area of the control points? 


ModelQlO 


Using the model, how difficult was it to plan your route? 


ModelQl 1 


Do you feel the model gave you an advantage you normally wouldn't have had? 


ModelQ12 


Would you use this tool if it were available for mission planning? 


ModelQlS 


Would you use this tool if it were available for mission rehearsal? 


ModelQ14 


Would you use this tool if it were available for navigation training? 



Table 4.4. Model Questions 

Participants had only moderate confidence in the trail network, obstacles, and 
vegetation representations in the model (Questions ModelQ3, ModelQS, and ModelQ6). 
This lead to participants having difficulty in identifying the controls once they were 
within 50m of the control (Question ModelQS) during the execution phase. This is due to 
the low-level vegetation (grass and brush) and smaller depressions of the actual course 
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that were not as detailed in the model. This is more apparent for controls located at 
ground level, Control Point 2 and Control Point 4, where more errors were made 
attempting to locate these controls than any of the other controls (Chapter IV, Section 
B.7). 

Participants were more likely to use the model for mission rehearsal (Question 
ModelQlS) than for mission planning or training of general navigation skills (Questions 
ModelQ12 and ModelQlS). Participants felt the model provided moderate assistance in 
planning their routes (Question ModelQll). It is interesting to note that even though 
their overall navigation performance was not as good as the map only participants, the 
VE participants had enough confidence in the model to use it for mission rehearsals. 
e. Interface Questions 
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Figure 4.25. Cell Bar Chart for Debriefing Questionnaire (Interface Questions) 



Code 


Question 


InterfaceQl 


Were you able to easily move through the model? 


InterfaceQ2 


Was the joystick easy to use? 


InterfaceQ3 


Was the acceleration lever easy to use? 


InterfaceQ4 


Were the toggle buttons easy to use? 


InterfaceQ5 


Your overall felling about the interface? 


InterfaceQ6 


Was the 15-minute train-up on the initial model useful? 


InterfaceQ? 


Was the 15-minute train-up on the initial model enough time to 
become familiar with the interface? 


InterfaceOS 


Did the use of three screens cause any confusion when maneuvering? 



Table 4.5. Interface Questions 
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As expected from their relative ease in moving through the environment and the 
limited tasks which the participant was asked to perform in the model, VE participants 
felt the model interface was user friendly (Questions InterfaceQl - InterfaceQ?). Also, 
the three-screen configuration provided them with no difficulties in viewing the 
environment (Question InterfaceQS). 

/. Model Needs 

The last page of the questionnaire asked participants to hst the items they felt 
would best assist them in navigating through a VE and real world. Streams and rivers 
were deUberately left off the list of possible water features to see if participants would 
pick these as linear features that should be portrayed in a VE. Findings are discussed by 
object groupings: buildings, miscellaneous objects, obstacles, roads, terrain, vegetation, 
and water. 



1) Buildings 




Figure 4.26. Cell Bar Chart for Model Needs (Buildings) 



Virtual environment participants were more discerning in the types of buildings 
that they wanted displayed in a model. They differentiated between more permanent and 
distinguishable buildings such as factories and public structures (churches, fire stations, 
schools, and government buildings) and the more abundant and ever changing structure 
of houses and shacks. This demonstrates the ability of VE participants to identify the 
more prominent landmarks in the model and disregard the less distinctive objects. 
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2) Miscellaneous Objects 

Nearly all the participants sought assistance from directional aids. A compass or 
virtual sun that can provide cues to direction of travel were the most requested items in 
this category. Map participants would also like to have location indicators such as street 
and road signs. Virtual environment participants viewed conplex items such as rock 
piles as being useful. This is due to the perception that if a model builder is going to put 
forth the effort to represent such a complex object, it must be a valid landmark. 

Cell Bar Chart 
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Figure 4.27. Cell Bar Chart for Model Needs (Miscellaneous Objects) 

All participants agreed that a model whose purpose is to provide spatial 
knowledge of an environment does not need to represent movable entities such as people 
rsd animals. Sound is also not seen as an essential need for most participants. This is 
r to the fact that moving entities and sounds provide little directional cues to the model 
- vr. The exception to this is the sound of a stream or highway noises. If these items can 
!•« spatially represented in the VE, they can provide navigational cues to the user. On 
weekends when training was being conducted at the Fort Ord Military Operations Urban 
Terrain (MOUT) Site, participants could maintain cardinal directions based on the sounds 
of weapons located southeast of the orienteering course. Sound based navigation cues 
were also provided by motorcycle or Formula One engines when races were held at the 
Laguna Seca Raceway which is also located southeast of the orienteering course. 
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3) Obstacles 
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Figure 4.28. Cell Bar Chart for Model Needs (Obstacles) 

Objects that are easily viewable from a distance or present a major inpact on 
mobility are highly requested in a VE. Participants requested vertical obstacles and 
elevated obstacles more often then smaller, more easily bypassed obstacles such as pits 
and shallow ditches. This is because the vertical and elevated obstacles are more 
permanent due to the difficulty to construct in the real world and because they can be 
easily seen and used as navigational aids. 

4) Roads 

Participants desire the representation of more permanent man-made linear 
landmarks such as roads. They are less prone to changes and provide a rapid means of 
travel through the environment. Roads also link more prominent landmarks which 
participants use for coarse movement (Appendix Q) through the environment. Trails also 
provide valuable information but, are more subject to change and are therefore less 
reliable. Footpaths change with the seasons and can be easily produced by man or animal 
and are less direct in their course through the environment. Footpaths may not lead 
participants to their intended destination which leads to a general distrust of paths that 
have not been traveled previously by the participant. 
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Figure 4.29. Cell Bar Chart for Model Needs (Roads) 

5) Terrain 

Terrain elevation plays a major part in determining one’s location in an 
environment. Participants desire adequate representation in changes of elevation. 
Terrain which is easily viewable from a distance (hills and ridgelines) was most often 
requested. A contradiction to this is the desire for spurs and fingers to be adequately 
represented but, not their compliment, draws. The two are distinguishable at a distance 
but, a lack of terrain elevation is not seen as important to the participants as the presence 
of elevated terrain. This is likely a result of no draws being present in the testing 
environment. 
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Figure 4.30. Cell Bar Chart for Model Needs (Terrain) 
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One of the major problem locations on the course was Control Point 4 which was 
located in a depression. Because of the difficulty many participants had with this control, 
many of them indicated the need for depressions to be adequately represented in a VE 
used to provide spatial knowledge of an environment. If a control point was located in a 
draw, more participants would have identified this as a type of terrain which must be 
adequately represented in the model. 

6) Vegetation 

Participants identified a lack of vegetation (a clearing) as a vital element of a VE. 
Participants understood that no matter how accurate the placement of vegetation in the 
environment, most of the trees and bushes are randomly placed and therefore should not 
be used as landmarks. Clearings, or the lack of vegetation, is seen as a more defining 
characteristic of a wooded environment, much as an oasis in a desert. It is the differences 
in vegetation which make the area distinguishable from its surroundings and useful in 
determining one’s location. 
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Figure 4.31. Cell Bar Chart for Model Needs (Vegetation) 

Undergrowth and flowers are seen as unnecessary fillers. This is in direct 
contradiction to the responses of the VE participants who felt the lack of grass and 
undergrowth in the model made it more difficult to locate controls placed at ground level 
(Chapter FV, Section B.4.d). This does indicate that VE participants realize that these 
items are not necessary for general navigation through an environment. Although 
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undergrowth may hinder cross-country movement, it provides little interference with 
visibility at elevations greater than three feet above the model’s surface. 

Since trees and brush do hinder visibility and movement through the environment, 
participants see a need for them to be represented in the environment. This type of 
vegetation provides an indication on which routes provide cover and concealment and 
which routes may make rapid cross-country movement impossible. 

7) Water 

Participants desire the representation of major bodies of water in the VE. The 
reasons are similar to those used for determining the types of roads and trails they would 
like represented (Chapter IV, Section B.4.f.4). Major bodies of water change relatively 
little compared to creeks and puddles that may be here one day and gone the next. 
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Figure 4.32. Cell Bar Chart for Model Needs (Water) 

Streams and rivers were not on the list of water objects provided to the 
participants (Appendix E.8). Two map participants added these to their list of objects 
that should be represented. Streams and rivers can provide the same types of 
navigational cues as paved and dirt roads; however, they are susceptible to course 
changes due to increased precipitation levels and soil erosion. 

8) General Conunents 

Map participants showed a greater desire for information than the VE and real 
world participants. They requested more objects be portrayed in the VE than the other 
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two groups. This indicates that VE and real world participants may have a better 
understanding of what landmark and defining terrain features are best used to navigate 
through the virtual world and can be easily identified during movement through the 
actual terrain. 

Although streams, rivers, and bridges were left off the list of potential model 
objects, they should be included in a virtual model. By their nature, they define 
boundaries within the environment and passageways between those boundaries. 
Although dismounted and mounted forces may be able to ford water obstacles, bridges 
still provide important links between sectors and due to their limited number and 
distinctive characteristics, they make excellent landmarks. 

9) Top Six Model Needs 

Objects on the Top Six Model Needs were assigned values based on their 
placement on each participant’s listing. Objects that were designated as the most 
inportant items were assigned a value of 6, the second most important item as 5, the third 
were assigned a value of 4, the fourth most important as 3, the fifth most important as 2 
and the least most important as 1. After assigning these values for each of the 
participant’s selections, the numbers for each object were added to determine their 
overall value. The objects were then ranked in order from highest to lowest totals and 
displayed on a bar chart (Figure 4.33). 

Terrain elevation is the major focus of most participants. Hills, ridgelines, spurs, 
and fingers rarely have major changes in their shape making them excellent navigation 
cues. The next most requested items were linear objects such as roads, trails, rivers, and 
streams. These objects help to identify boundaries within our environment and provide 
directional cues. Man-made structures were next on the list of items desired by the 
participants. Trees and clearings were included but, not placed high of the list of needs. 
Other directional tools such as compasses were also noted. The frequency of items 
included on the top six list bears a striking resemblance to the order in which the objects 
were placed on the model developed for this experiment (Chapter lH, Section C). 
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Figure 4.33. Cell Bar Chart for Model Needs (Top Six Model Needs) 

Using the items identified as the most essential for use in a virtual environment, a 
stripped down terrain model could be created which portrayed only elevation changes, 
linear features, and landmark models. Vegetation could be represented by color coding 
the terrain skin like a map or placing colored walls indicating the type of vegetation and 
its height. Based on the results of this experiment, one would expect that the symbolic 
nature of the VE would assist participants in identifying prominent landmarks while 
reducing the confusion created by diverse and dense vegetation. This could help to focus 
participants on key features and enhance navigational performance. 

6. Simulator Sickness 

During the experiment, a tendency for simulator sickness showed for the one pilot 
and two VE participants who attempted to run a clean route through the model. These 
participants were all able to make it past Control Point 4 before stating they felt ill. Non 
of the participants made it past Control Point 6 before they had to stop and leave the 
room. Participants were given a time credit if they felt sick and had to step away from 
the model. After a five to ten minute break, the participants returned to the model. None 
of them were able to make it past Control Point 7 before feeling ill and stopping their use 
of the simulator. 
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The simulator sickness could be due to one or more factors. When the frame rate 
of the model is less than 30Hz [PAUS 92], and the screen refresh rate is less than 70Hz 
for color monitors [BAIL 89], equilibrium problems may occur. The problems occur as 
individuals identify one rate of motion with their peripheral vision while other neural 
processes perceive a different rate of motion [VAN 90] [EBEN 92]. The critical fusion 
frequency is achieved when the refresh rate has reached a level where a steady image is 
attained; normally this is approximately 60Hz [POLE 97] [THOM 97]. However, this 
rate can fluctuate plus or minus 20Hz depending on the individual [ROGO 83]. 

The spinning of the billboards and the popping of the forest walls may also have 
played a role. As participants moved through the model, the trees would rotate as they 
passed by the participants’ heads. This may have caused equilibrium problems between 
the middle ear and the optical cues. A participant running a final route also concentrated 
more on the model, taking fewer breaks to look at the map. The constant staring at and 
motion of the model may have played a factor in participants succumbing to simulator 
sickness. 

The fact that none of the participants was able to complete a clean run of the VE 
may have impacted their ability to complete the actual course with limited errors and map 
checks. The best performance of any VE participant was by Virtual Environment 
Participant #1 who reached Control Point 7 before stopping the use of the VE due to 
simulator sickness. 

7. Distinguishing Terrain Elevations 

During the execution of the course, all participants showed some difficulty with 
locating Control Point 4 (Chapter IV, Section B.13). For most participants, this difficulty 
arose as they misread the map and thought the control point was located on a lull instead 
of in a depression. This same problem was apparent in all participant groups. The map 
only participants demonstrated this problem the most. Real World participants 
encountered this problem during the training phase and one participant was never able to 
overcome the error in time to locate the control point. Surprisingly, two of the VE 
participants also displayed this problem, both of whom failed to locate CP4 during the 
evaluation phase. These same participants had difficulty locating the control in the VE. 
The exploration of the environment by the real world participants and virtual 
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environment by the VE participants placed most of the participants in the correct vicinity 
of Control Point 4 unless a participant took the wrong trail enroute to the low ground. 

This indicates that identifying the difference between hills and depression on a 
map is difficult for some individuals. With a time-compressed study of the map and the 
environment, many people failed to properly identify depressions. Study of a VE or the 
actual terrain may assist in identifying the improper interpretation of the contour lines 
and helps individuals construct a more accurate three dimensional representation of the 
terrain for their own mental map. 

8. Need for Land Marks to Locate Control Point 

Participants who recognized that Control Point 4 was in a depression and not on a 
hill encountered problems pinpointing the control since it was in a shallow pit surrounded 
by knee high grass and brush. The dense low-level vegetation, positioning of the control 
point below ground level, and the limited landmarks in the vicinity of the control point 
made locating the control difficult. The participants who used the lone tree 17.5m to the 
west or the jetty of brush 21m to the south of the control as a landmark had the least 
difficulty locating Control Point 4. Real world Participant #5 was unable to locate CP4 
during the training phase but, realized before he underwent the execution phase that if he 
went to the tree 17.5m west of the control and worked his way back, he would have better 
luck in locating the control. He implemented this strategy during the execution phase and 
walked straight into Control Point 4. 

Control Point 2 was also positioned below ground level. Real world participants 
showed difficulties locating this control point (Chapter IV, Section B.13) although a very 
distinctive landmark, a shed, was located less then 20m to the east. Most participants 
who showed difficulty with this control point veered to the north of the flag and searched 
the terrain to the north and west. Participants who used the shed as an anchoring point 
and foUowed the edge of the trees to the south of the shed had little to no difficulty 
locating the control. 

Control Point 7 was positioned below ground level in a trench line. Participants 
who made it this far showed httle difficulty locating the control. Most participants 
approached the control from the east and walked straight into the end of the trench line. 
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Other participants intersected the trench somewhere to the west of the control and 
followed the trench to its east end where the control was located. 

The positioning of controls below ground level made them difficult to locate. 
However, the successful use of landmarks in the vicinity of the controls made the 
controls easier to locate. Participants who failed to recognize or utilize easily identifiable 
landmarks found themselves confused and off course. The more distinct the landmark, 
the easier it was for participants to fix their position in the vicinity of the control and 
develop a search plan to locate the control. Submerged controls located at the end of 
linear landmarks or close to very distinct landmarks were easier to locate than submerged 
controls located in areas with limited or indistinct landmarks. 

Effort should be taken to identify and replicate easily identifiable landmarks in the 
VE to assist in locating items or fixing user positions in areas that can be confusing or to 
assist in locating objectives that are well concealed. If not, confusion will occur while 
navigating in the VE which can transfer over to problems with navigating in the actual 
terrain. 

9, Correlation Between Disorientation in Virtual Environment and 
Disorientation in Real World 

A review of training and execution phase routes for the VE and the real world 
participants indicates a possible correlation between the locations individuals where 
disoriented during the training phase and the locations they became disoriented during the 
execution phase. Most participants showed difficulty in maneuvering between Control 
Point 3 and Control Point 4. This was the first leg that required participants to traverse a 
straight-line distance of more than 300m. 

Further research is needed to determine if there is a direct correlation between 
locations individuals become disoriented in the VE and were they become disoriented in 
the real world. If there is a link between the two, VEs can be used to validate mission 
routes and to assist decision-makers in predicting mission success probabilities. Virtual 
environments could also be used to identify trouble spots to bypass, conduct map checks, 
or disambiguate landmarks to ensure individuals do not become lost. 
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10. Banker Participants vs Goei^er Participants 

a. Differences in Models 

The model used in MAJ Banker’s experiment [BANK 97] was a non-real 
time representation of the environment developed using a golf course creator tool. The 
tool allowed for a very detailed model that had many characteristics of a map. The 
surface of the model was colored in a fashion that produced clean edges between the 
different types of vegetation. This gave the surface a map equivalent characteristic and 
provided the users with the ability to easily distinguish the difference between forested 
areas and clearings. The differing grass colors produced an effect similar to moving from 
covered terrain to open terrain as it delineated region changes. Since the model was 
developed using a golf course tool, users had to select which portion of the course they 
wished to explore and teleport between holes in order to view different portions of the 
model. This was not a seamless transition, as it required participants to refer to a master 
layout to determine which golf course hole and orientation they deeded displayed. 

The model used in this experiment was a real time representation of the 
environment in which users could seamlessly traverse the entire course area. The 
relatively uniform color of the model surface covered with an aerial photograph helped to 
render shadows but, provided no sensation of having an overhead canopy. Appendix P. 1 
outlines some of the other differences between the two models, the real world, and the 
map only training conditions. 

b. Similarities in Performance 

To make comparisons between the two experiments, participants’ experience 
levels for this experiment were reclassified (Appendix 0.2) in accordance with 
MAJ Banker’s participant experience level criteria [BANK 97]. Although performance 
levels were lower in this experiment compared to the Banker experiment, relative 
comparisons can be made between training conditions and ability groups for the two 
experiments. Similarities are shown for Total Error Distance by Ability Group (Figure 
4.34), Distance Per Error by Ability Group (Figure 4.35), and Map Checks by Ability 
Group (Figure 4.36). This indicates that MAJ Banker was correct in his conclusions that 
ability group has a limited impact on a participants ability to recognize and recover from 
an error. 
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Figure 4.34 displays Total Error Distances based on Banker Ability Groups for 
participants in this experiment with a lower score indicating better performance. The 
results do not indicate statistical significance between the two groups, F(l,13) = 1.702, 
P = .2147. Direct observation suggests that participants in the Beginner Ability Group 
traveled further off their planned route than the Intermediate participants who were better 
able to maintain their planned course. This fact is relevant only when viewed in context 
with errors committed or total route distance. A participant who traveled 300m off their 
planned route and crossed over 3000m of the planned course, performed better than a 
participant who traveled 2000m, 250m of which was off their planned route. 



Box Plot 

Grouping Variable: Banker Ability Group 




Figure 4.34. Interaction Box Plot for Total Error Distance (Banker Ability Group) 
Figure 4.35 displays Distance Per Error based on Banker Ability Groups for 
participants in this experiment with a lower score indicating better performance. The 
results do not indicate statistical significance between the two groups, F(1.13) = 1.847, 
P = .1973. Direct observation suggests that participants in the Beginner Ability Group 
traveled further per error committed than Intermediate participants who were better able 
to identify when they had deviated from their planned route. Beginner participants had a 
more difficult time recognizing their errors, fixing their position and orientation in the 
environment, and developing strategies to recover from their errors. This is to be 
expected since their navigation skills were limited in comparison to the Intermediate 
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navigators who had a better understanding of the task and a higher level of confidence in 
their skills. 



Box Plot 

Grouping Variable; Banker Ability Group 




Figure 4.35. Interaction Box Plot for Distance Per Error (Banker Ability Group) 



Box Plot 

Grouping Variable: Banker Ability Group 




Figure 4.36. Interaction Box Plot for Map Checks (Banker Ability Group) 



The Map Checks of the two conditions is shown in Figure 4.36 with a lower score 



indicating better performance. The means between groups does not indicate a statistical 



difference, F(l,13) = .447, P = .5154. Direct observation suggests that participants in the 
Intermediate Ability Group had a greater variance in the number of map checks 
performed per participant than Beginners. This, in conjunction with the number of errors 
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committed, indicates that many intermediate participants performed maintenance map 
checks to ensure they were still on their planned route. 
c. Differences in Performance 

Although MAJ Banker’s thesis did not discuss the following measures, there is a 
difference in performance between participants in his study and participants in this study 
for Controls Attempt (Figure 4.37), Controls Found (Figure 4.38), Errors Per Control 
Attempted (Figure 4.39), and Distance by Training Condition (Chapter IV, Section 
A.4.b). Measurements for MAJ Banker’s participants (Appendix P.2) indicate a more 
level performance across ability group, where this study indicates better performance by 
individuals rated as intermediates over those rated as beginners by the criteria outlined in 
MAJ Banker’s experiment. Figure 4.37 displays the Controls Attempted based on Banker 
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Figure 4.37. Interaction Box Plot for Control Attempt (Banker Ability Group) 
Ability Groups for participants in this experiment with a higher score indicating better 
performance. The results suggest a statistical significance between the two groups, 
F(l,13) = 5.226, P = .0295. The graph implies that participants in the Intimidate 
Ability Group are more likely to attempt a control than participants who are classified as 
Beginners. 

Figure 4.38 displays the Controls Found based on Banker Ability Groups for 
participants in this experiment with a higher score indicating better performance. The 
results indicate statistical significance between the two groups, F(l,13) = 5.987, 
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P = .0295. The graph suggest that participants in the Beginner Ability Group are less 
likely to find as many controls as participants who are classified as Intermediates. 
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Figure 4.38. Interaction Box Plot for Controls Found (Banker Ability Group) 
Unlike the Banker study, when comparing controls found by treatment group and 
Banker Ability Group, the Intermediate VE participants did not locate more controls than 
their real world and map participant counterparts (Figure 4.39). The plot indicates no 
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Figure 4.39. Interaction Bar Plot for Controls Found 
(Treatment and Banker Ability Group) 
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significance between training conditions based on Banker’s Ability Levels. The 
difference in performance between Banker’s results and this experiment’s findings is due 
to the more simplistic nature of the Banker model. Since participants could not move 
quickly through Banker’s virtual environment, they were forced to focus their efforts 
when using the model (Chapter IV, Section B.lO.d). In other words, for the Banker 
study, the VE and map groups were nearly identical. If intermediate VE participants in 
this study would have focused their efforts around control points and major decision 
points utilizing the teleport option and top down view more, their ability to find control 
points would have been more comparable to Banker’s participants. 

The Errors Per Controls Attempted of the two conditions is shown in Figure 4.40 
with lower score indicating better performance. The means between groups does not 
indicate a statistical difference, F(l,13) = 3.757, P = .0746. Direct observation suggests 
that participants in the Intermediate Ability Group make fewer errors per control attempts 
which implies they are better able to stay on their planned routes than Beginners. 



Box Plot 

Grouping Variable: Banker Ability Group 
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Figure 4.40. Interaction Box Plot for Errors Per Controls Attempted (Banker Ability Group) 
Similar to the comparison of controls found by treatment group and Banker 
Ability Groups, the Intermediate VE participants did not attempt more controls than their 
real world and map participant counterparts (Figure 4.41). The plot suggests no 
significance between training conditions based on Banker Ability Levels. Once again, 
the difference in performance between the Banker’s findings and this experiment’s 
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results is due to the more simplistic nature of the Banker model (Chapter FV, 
Section B.lO.d). 
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Figure 4.41. Interaction Bar Plot for Controls Attempted 
(Treatment and Bankei Ability Group) 

An identical 2-way ANOVA was run comparing Map Check Score and Total 
Error Distance by treatment group and Banker Ability Groups. Unlike the Banker 
experiment, no statistical significance was shown between training conditions based on 
Banker Ability Levels for Map Check Scores, ]F(2,9) = 1.602, P = .2539. Nor was there 
any statistical significance shown between training conditions based on Banker Ability 
Levels for Total Error Distance, F(2,9) = 1.522, P = .2695. The Banker study found that 
his VE d a statistically significant increase in performance in these two areas for 
intermediate participants. Banker’s findings were not supported by this study. This is 
due to differences in the type of navigation experience between the two participant pools. 
d. Reasons for Performance Differences 
As a whole. Banker’s participants performed better than the participants of this 
experiment. This is due to differences in the participant pools and the structure of the 
experimental design. MAJ Banker’s participants were more experienced in sports 
orienteering and the use of orienteering maps than participants of this experiment. They 
were also more familiar with running orienteering courses in typical central California 
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coastal terrain since many of them had participated in events in the San Francisco Bay 
area, Santa Cruz Mountains, and Monterey Peninsula. This resulted in a difference in the 
type of experience each intermediate group possessed and the kind of information they 
were able to extract from the VE, map, and natural environment. Due to the inexperience 
of many of the participants in this experiment, their planned routes were more difficult 
than those in the Banker experiment. If this experiment’s participants had more 
experience with navigating in central California coastal lands, they would have been able 
to extract more pertinent information from the study materials and improved their overall 
navigational performance. This indicates a possible use of virtual environments to train 
navigation skills for areas which soldiers may not routinely encounter. If generic 
navigation skills can be taught through the use of generalized terrain models for desert, 
arctic, mountainous, jungle, and wooded terrain, the VE would provide a useful tool for a 
commander’s training program. 

The experimental outline of MAJ Banker’s thesis was less intrusive to the 
participants. AH pointing tasks were conducted prior to the execution of the planned 
route. Once the planned route was initiated, no planned interruptions were made. For 
this experiment, pointing tasks were interjected at Control Points 2 and 4. This 
interrupted the flow of the planned route and may have had an effect on participant recall 
of their planned route. However, most participants showed little difficulty in navigating 
to the control points immediately following the Wheel Tests. Participants also received 
assistance from monitors if they were off course for more than 15 continuous minutes and 
were not making progress towards their designated control (Chapter IV, Section A.4.a). 

The poorer performance of this experiment’s VE participants is due to their 
reduced ability levels and the ability of MAJ Banker’s Non-Real Time VE to provide an 
exocentric as well as an egocentric view simultaneously. This reduced the effort required 
by participants to locate themselves in the environment since the computer model 
resolved this issue for them. Since the model was a non-real time model, participants 
teleported between control points and decision points in the environment instead of 
conducting cross-country movement through the model. Participants in the real time 
model had to navigate through the VE and determine their location as they moved. This 
required them to utilize the navigation cycle (Appendix Q) during the study phase and 
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use training time to move through terrain which provided few disambiguating features to 
assist them with navigating through the actual terrain. If participants had used the real 
time VE to explore the area around the control points and to identify the differences 
between key decision points along their planned route, their performance would have 
improved. 

11. Map Resolution 

The map used for this experiment far exceeds the capabilities of most maps used 
during traditional military operations or orienteering competitions for its level of detail. 
Any dismounted infantry or special operations soldier would treasure a map of such 
detail when entering into a new area of operations. Most military operations maps are at 
a scale of 1:50,000 or 1:24,000 (Appendix F). 

Because of the small scale of the map and the use of orienteering terrain 
classification markings, participants could glean information from the map that would 
normally not be available to them. Most participants gave the map above average ratings 
for clarity, information provided, and ease of use (Chapter IV, Section B.4.a). The 
increased detail provided participants with enough information that they did not need to 
use the VE or real world to discover and catalog many landmarks or changes in terrain 
elevation that would normally be too indistinct to appear on most maps. This resulted in 
map group participants performing much better than would normally be expected of an 
individual who was provided with only a map, objective photos, and objective locations. 

12. Resolving Ambiguities In Mental Maps 

When participants began the course, they first had to identify their position and 
orientation (Appendix Q). Once this was accomplished, they checked their mental maps 
and list of instruction to determine their course of action. Once movement began, they 
were continuously cycling through a series of mental processes to verify their position, 
orientation, mental map, and route. While updating their mental maps, each group was 
faced with a unique set of issues. 

Map participants tended to navigate using propositional knowledge, a list of 
directions, that when linked together would lead them through the course. This 
represents • exocentric knowledge of the environment that is spatial but, not temporal 
which meant that the fidelity of the environment did not encumber map participants. 
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Often they concentrated on more definable characteristics of the course such as buildings 
and trail intersections as well as the distance between them, rather than less 
distinguishable objects such as trees. This required the translation of propositional 
knowledge into ambiguous static imagery (Appendix Q). Since their imagery was 
mdistmct, map participants did not panic when their mental maps did not match the 
actual imagery they encountered. Participants did have to resolve the differences 
between perceived distances on the map and actual distances on the ground as well as 
visualizing the different categories of terrain depicted on the map. Participants who paid 
close attention to the first couple of trails they encountered during the initial portion of 
the course, quickly resolved this issue and showed little difficulty with perceived 
distances for the rest of the experiment. 

Real world participants were faced with a different set of issues as they attempted 
to resolve differences in their perceptions of the real world and the actual terrain they 
were standing on. Similar to the map participants, those real world participants who paid 
close attention to the first couple of trails they encountered during the initial portion of 
the course, quickly resolved the distance issues and showed little difficulty with 
perceived distances for the rest of the experiment. Those real world participants who 
failed to resolve this issue during the training phase showed difficulties with judging 
distances during the middle portion of the course. 

Although real world participants had traveled the terrain once before, their 
perception of the environment was based on dynamic imagery (Appendix Q). Dynamic 
imagery is similar to a mental movie. The navigational performance of real world 
participants is based largely on how well they developed their mental movie during their 
initial run through the environment. If participants make it only partially through the 
environment or become confused along their route, their movie becomes a poorly edited 
collection of three-dimensional clips. The movie remains that way until they can make a 
clean run through the course editing their movie, clarifying discrepancies as they move 
through the environment. Real World Participant #1 demonstrated difficulties in editing 
his mental movie due to the inordinate amount of time he spent looking for Control Point 
3 during the training phase. He searched the area between Control Point 2 to the western 
boundary looking for Control Point 3. Because of this, his mental map and d 5 mamic 
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imagery of the environment was cluttered with ambiguous representations. As a result, 
he was unable to locate Control Point 2 without crisscrossing the area between Control 
Point 1 and Control Point 3 four times. Participants with well edited mental movies 
performed movement along their planned route better than participants who had poorly 
edited depictions of the environment. During the training phase, real world participants 
showed the same difficulty with resolving distances that the map participants initially 
encountered during their execution phase as they dealt with resolving differences in 
propositional knowledge with the actual terrain. 

Virtual environment participants had to deal with many of the same issues which 
the real world participants faced in resolving inconsistencies with their dynamic imagery 
of the environment and their perceptions of distance. Misperceptions of distances and 
sizes from use of the VE caused many VE participants to over estimate the distances they 
need to travel when operating on the actual terrain. The placement of a HMMWV at the 
start point of the VE did not appear to alleviate these perception issues. Virtual 
Environment Participants #1, #2, #3, and #5 all commented on their initial difficulty with 
resolving distance during the execution phase of the experiment. Similar to the map and 
real world participants, those VE participants who paid close attention to the first couple 
of trails they encountered during the initial portion of the course quickly resolved the 
distance issues and showed little difficulty with perceived distances for the rest of the 
experiment. 

To compound the issues of perception, VE participants had to resolve differences 
between the model environment and the actual terrain. Differences in vegetation density 
and complexity as well as perspective issues required additional mental manipulations in 
the environment. Participants who interpreted the vegetation in the VE as symbolic 
representations of vegetation did not regard vegetation as a landmark object and were 
able to disregard the vegetation while moving through the actual environment. This 
allowed them to focus on more distinguishable landmarks. 

Participants are willing to accept oversights with maps and VEs, but, are 
unwilling to accept errors (Figure 4.42). If the map or VE is missing something that is 
present in the real world, individuals accept this as changes in the environment since the 
model or map was developed. In other words, the map and VE are assumed to be a 
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subset of the real environment. They resolve the differences with their mental map by 
adding the missing feature. If something appears on the map or in the VE, individuals 
expect to see that same object or feature in the real world. When individuals do not see 
the feature in the real world, confusion may result and they question the validity of the 
map or VE. This can result in total mistrust of the mental map created from a physical 
map or VE. 




Figure 4.42. Venn Diagram of Real World Abstractions 
It is better to leave something off of a map or out of a VE than to misrepresent it. 
Misrepresentation of objects or features on maps and in VEs creates confusion and 
mistrust which leads to individuals abandoning their mental maps and questioning their 
knowledge of the environment. Ultimately this results in a large drop in navigational 
performance. Exposure to maps still provides ambiguity issues. Maps are excellent at 
providing propositional knowledge that produces superior static imagery for route 
knowledge with one caveat. Static imagery is easily fooled by parallel errors (Chapter 
rV, Section A.4). Map Participant #3 felt his biggest problem was resolving distances. 
This lead to his parallel errors at Control Point 2 when he was searching the clearings to 
the north of the budding instead of to the west. With a more dynamic image of the area, 
he would have been able to recognize the difference in the clearings such as the open area 
to the south of the clearing in which CP2 was actually located. Map Participant #4 
experienced parallel errors in locating Control Point #3 since from the map study, he was 
unable to distinguish the differences between the buildings located in the vicinity of CP3. 
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The exposure to the actual environment or an accurate VE can help to resolve 
these ambiguities by filling in the gaps by changing static imagery into dynamic imagery 
which can clarify discrepancies that lead to parallel errors. Virtual Environment 
Participant #1 commented that he wished he had used the model to study the area more 
carefully around Control Point 4 and Control Point 9. This would have helped him 
disambiguate the terrain at these locations. Virtual Environment Participant #2 gained 
tremendous confidence in his location when he saw the building near Control Point 2. As 
he approached the area from the north, he stated, “I know from the model that I have to 
go left at this broken down building to find CPI.” During the debriefing, he commented 
that he remembered the pavilion near Control Point 3 from the model which helped him 
fix his position when searching for the control. 

13. Map Checks vs Errors 

MAJ Banker briefly discussed the correlation between map and compass checks 
and distance off route [BANK 97]. His results indicated that the further a participant was 
off the planned route the more likely the participant was to conduct a check. A similar 
simple regression was performed in this experiment to see if the same findings would 
hold true. For this analysis, the Normalized Average Distance Per Error (Chapter IV, 
Section A.4.b) and the Normalized Map Check Score (Chapter IV, Section A.4.c) for 
each participant was used. These two measurements were used because the 
measurements are normalized to take into account the number of errors committed and 
the number of control attempted. 

The results were comparable to the Banker study. The results show a direct 
correlation between the distance participants were off their planned route and the number 
of checks they performed, F(l,13) = 32.380, P = .0001 (Figure 4.43). The results indicate 
that participants who are on their planned route are more confident in their performance 
and conduct fewer checks. Once participants recognize they are off their planned route, 
they conduct checks in accordance with the distance they have deviated from their 
planned route. The further participants were off their planned route, the more checks it 
took them to return to the proper route. 
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Normalized Map Check Score (Attempted) 



Y= 12.141 + 35.122 * X; R'^2= .714 

Figure 4.43. Regression Plot for Total Error Score Per Control Attempted vs Normalized 

Map Check Score 

14. Average Distance Off Route 

In MAT Banker’s thesis, he briefly reviews the distance participants veered off 
their plarmed route per control point [BANK 97]. His conclusions indicated that the real 
world participants’ performance declined as they traveled further through the course 
because they were unable to explore their entire routes during the training phase due to 
the one-hour time constraint. Similar conclusions were drawn from this experiment 
based on the fact that only two of the real world participants were able to make it further 
through the course on the execution phase than on the training phase. This issue can be 
addressed by time compressed training in a VE that allows individuals to more rapidly 
explore an environment through increased speed of movement or teleportation. 

Making comparisons to MAJ Banker’s scatter plot of Treatment Group Distances 
Off Route by Control [BANK 97] and Figure 4.44, we see an interesting correlation 
between the average distance per participant on Control Point 4. The notably marked 
increase in average distance per participant indicates that Control Point 4 presented 
participants of both studies with a more difficult task. The reasons for this were 
discussed previously (Chapter IV, Sections B.6 and B.7). 
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Scatter Plot 




Control Pmnts 

Figure 4.44. Scatter Plot of Average Distance off Route Per Control Attempted 
Although Figure 4.44 indicates VE participants having even more difficulty with 
Control Point 4 than the real world and map only participants. Figure 4.45 indicates that 
the difficulty with the control is actually more uniform across the groups. This scatter 
plot displays a normalized distance per error. 




— ♦— Map 

" ■'■■■' ii ' Virtual Bn v 

- -* - Real World 



Figure 4.45. Scatter Plot of Average Distance Per Error Per Control Attempted 
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Both graphs indicate a more level performance after Control Point 4. However, 
only 11 participants made it past Control Point 4 with only 66% of the participants 
making it past Control Point 6, and only the better navigators making it through the 
course. The plot shows that once an error is committed, the mean performance for 
participants demonstrates the same amount of difficulty in recovering fi-om the error for 
Control Point 4 regardless of training condition. 

Figure 4.45 also indicates that map and VE participants had more difficulty 
recovering from errors during movement from the starting point to the first control. This 
is due to both groups of participants attempting to resolve resolution differences between 
their mental maps and the real world (Chapter IV, Section B.ll). Virtual environment 
participants showed a marked improvement in their ability to recover from their errors 
from Control Point 1 to Control Point 3, but, ran into difficulty with Control Point 4. 
After Control Point 4, performance for all remaining participants leveled off except for a 
decrease in performance at Control Point 7 for the map participants. The real world 
participants demonstrated the most difficulty of all participants who executed Control 
Points 5 through 9. This supports MAJ Banker’s conclusions that real world participants 
have more difficulty with the later stages of the course because of their inability to 
traverse this section of the terrain during the training phase. 
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V. CONCLUSIONS AND RECOMMENDATIONS 

A. CONCLUSIONS 

1. General Conclusions 

This experiment studied the effects of training methods on spatial knowledge of a 
natural environment given a one-hour exposure to a high resolution 1:5,000 orienteering 
map, access to the orienteering course, or a high fidelity real time virtual representation. 
The following conclusions are drawn from both the quantitative and qualitative results: 

a. Experiment training conditions show no significant effect on an 
individual’s ability to obtain and demonstrate spatial knowledge of a natural 
environment (Chapter IV, Section A.6). 

b. Spatial ability plays a significant role in an individual’s ability to 
obtain and demonstrate spatial knowledge of a natural environment (Chapter IV, 
Section B.4). 

c. Exposure to the actual terrain or a virtual representation of the terrain 
eliminates ambiguities in an individual’s mental map by providing dynamic 
imagery to disambiguate propositional knowledge gained from maps (Chapter IV, 
Section B.12). However, there are other issues with walking the ground or VE 
that prevent these training tools from being better than a really good map for short 
exposure durations. 

d. A high resolution 1:5,000 orienteering map provides an inordinate 
amount of detail which is uncommon in typical maps used for military operations 
and has a significant effect on an individual’s ability to obtain and demonstrate 
spatial knowledge of a natural environment (Chapter IV, Section B.ll). It’s hard 
to beat a really good map. 

2. Performance by Study Group 

Based on the use of a 1:5,000 orienteering map and a high fidelity real time VE, 
the results suggest that provided with an hour to study the environment and plan a route, 
map only participants gained more spatial knowledge of a 1km square piece of terrain 
than VE participants. VE participants performed on par with real world participants. 
Overall con^arisons of results indicate that map participants outperformed real world 
participants who outperformed VE participants in route knowledge. Results suggest that 
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VE participants had slightly better survey knowledge than map and real world 
participants. 

3. Performance by Spatial Ability 

Using a high resolution 1:5,000 orienteering map and a high fidelity real time VE, 
results suggest that provided with an hour to study the environment and plan a route, 
participants with above average spatial ability scores gain more spatial knowledge of a 
1km square piece of terrain than participants with below average spatial ability scores. 
Overall comparisons of results indicate that participants with above average spatial 
ability scores outperformed those with below average spatial ability in route knowledge. 
Results suggest that above average spatial ability participants had better egocentric 
knowledge of the environment and slightly better excocentric knowledge than below 
average spatial ability participants. 

4. Mental Maps 

Behavioral analysis in this experiment suggests that the earlier models of 
navigation based on the assumption that individuals, while navigating, make decisions 
depending on a corrqjarison between what they see versus what thy expect to see 
[PASS 84] [JUL 97], are only partially correct. Virtual environments and the real world 
provide us with mental imagery of our environment. Maps only provide us with 
propositional knowledge that most people translate into an ambiguous series of static 
menttil images as they move through the environment. Exposure to VEs or the real world 
assists individuals in filling the gaps, resulting in dynamic mental imagery, 
disambiguating images created by the study of a map. With enough exposure to the real 
world and VEs, we can rescive any discrepancies in our mental imagery which enhances 
our confidence in our navigation through the environment. An individual’s preconceived 
egocentric view, whether developed from the study of the real world or VE, provides a 
strong mental image. A participant cannot help but refer to this mental image, even if it 
is incomplete or confused. 

5. Map Resolution 

The performance of the map only subjects may have been skewed based on the 
scale and resolution of the map used for the experiment. This map (1:5,000 orienteering 
map), provided more information than is usually available on maps used for most military 
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operations. Providing participants with a map of less resolution will surely produce 
worse results for the map only group and improve the perceived performance of the VE 
and real world groups. 

B. SIGNIFICANCE 

1. Study Method 

Given a limited timeframe and highly detailed maps, individuals will gain more 
information about the target environment than subjects afforded high fidelity real time 
models. Given less than an hour time to prepare for a mission, individuals should 
concentrate more on map study and route memorization than general terrain 
familiarization if they wish to maximize their performance. This fact helps to limit 
expenditure of resources and reduces the possible confusion of mission forces, allowing 
them to focus on those assets which will best assist them in preparing for the operation. 

2. Spatial Ability 

Identifying individuals with above average spatial ability will assist in predicting 
which personnel may be better suited as navigators. If these visual and organizational 
traits can be identified and taught, it will assist in training individuals to perform 
navigation tasks in a manner which will improve their spatial knowledge and overall 
performance in the area of operations. 

3. Mental Imagery 

Giving a relatively unconstrained timel in e for mission preparation, individuals 
can make numerous runs through a VE. This will allow individuals to edit their mental 
movies providing them with an excellent three-dimensional mental representation of the 
environment to assist them during navigation. 

4. Map Resolution 

Maps with high resolution provide a tremendous amount of information. If 
individuals have a limited amount of time to prepare for a mission, a high-resolution map 
is a better tool for gaming spatial knowledge of an area. The same detail required to 
produce a high fidelity VE is the same information required to produce a high resolution 
map which depicts vegetation densities and building orientations. Producing a high- 
resolution- map is less time intensive and results in a two dimensional representation 
which is easily carried and studied. With a limited amount of time for mission 
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preparation, resources should be placed on producing a high-resolution map that can be 
quickly mass reproduced and distributed to the mission force. This will save precious 
resources while affording individuals with the best training tools for the available time. 

C. CONTRIBUTIONS 

The experiment implemented two tests that were developed to help test an 
individual’s excocentric and egocentric knowledge of the environment. These tests were 
essential in determining if individuals had gained only route knowledge of the 
environment or if they were able to obtain and demonstrate survey knowledge. 

1. Wheel Test 

To measure egocentric knowledge of the environment, the Wheel Test was used. 
This test is a variation of the one initially developed for use in an experiment conducted 
to test the transfer of spatial knowledge from a VE to a con^lex man-made structure 
[GOER 98]. The test measures an individual’s ability to identify the direction to several 
locations within the environment without providing the individual with directional cues 
(Chapter HI, Section G). It requires that individuals understand their relative position 
within the environment with regards to locations they have or will be visiting while 
navigating through the environment. Variances from actual measurements are calculated 
and combined for comparison with other participants. 

2. Whiteboard Test 

To measure exocentric knowledge of the environment, the Whiteboard Test was 
used. This test is also a variation of the one initially developed for use in an experiment 
conducted to test the transfer of spatial knowledge from a VE to a con^)lex man-made 
structure [GOER 98]. The test measures an individual’s ability to identify the relative 
position of the control points to one another without any external reference cues (Chapter 
in. Section G). It requires that individuals understand the relative position of the controls 
to each other without worrying about a distance scale. Angles between each control point 
are measured and used for conparison with the actual measurements and to compare 
against other participant results. 
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D. FUTURE WORK 

1. Displays 

During the experiment, participants were exposed to a large three-screen display 
with over 103° field of view and 4800 square inches of viewable surface. This is done to 
provide the participant with the largest viewable surface possible. Little research has 
been conducted to determine which type of display provides the best possible 
environment for navigating. Single screen, triple screen, head mounted display (HMD), 
and wide screen views are just a few options available which are easily configured for 
use at most simulation centers. 

Usability participants in this experiment and in Sullivan’s helicopter navigation 
experiment [SULL 98] indicated that the wider field of view dramatically assisted in 
navigating through the environment. Participants felt they were able to extract 
navigational cues firom the terrain on their peripheries much as they would do in the real 
world. This suggests that a wide screen or three-screen configuration may be the best 
display for such training and mission preparation devices. However, these participants 
were not exposed to either environment using a HMD. 

Some users appear to be more prone to simulator sickness than others. This can 
be due to many factors from direct e^qjosure time to refresh rate (Chapter IV, Section 
B.6). During this experiment, participants who made it through the environment and 
were conducting a final run, expressed a feeling of simulator sickness between Control 
Point 4 and Control Point 5. Each of these participants stopped their movement and took 
a five to ten minute break before going back into the simulation. None of these 
participants completed their second trip through the model as they all stopped somewhere 
between Control Point 6 and Control Point 7 even though they had time remaining. 

Further research is needed to determine which display and frame rate are best to 
reduce the instances of simulator sickness while providing the user with the best possible 
display for obtaining the information needed to maneuver through the environment. 

2. Interfaces 

Training and mission preparation occurs in many locations and in many forms. 
From initial training in classroom type environments to last minute revisions while flying 
to the mission release point, soldiers are continuously planning, revising, and rehearsing 
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the mission at hand. Because of this, the interface for conr^uter training devices must be 
customized to the task to provide the best possible results. However, we don’t know 
what the task to interface relationship is. 

To the user, the interface is the system and therefore the interface must optimize 
system utility while limiting factors that may hinder performance [HIX 93]. Whether it is 
a keyboard, joystick, mouse, data glove, Polhemus device, locomotion device, or some 
other interface, the device must be easy to use and provide the user with the versatility 
needed to maneuver through the environment to provide positive training transfer. 
Anything less will reduce the effectiveness of the training and could foster negative 
training effects. Continued research is needed in this area to determine what type of 
interface is best for simple navigation through a VE. 

3. Fidelity Levels 

As we increase the capabilities of our hardware and software, the definition of 
low, medium, and high fidelity models become more diverse. In the early 1970’ s, people 
were impressed with the wonderful new game played on televisions called Pong. Today, 
home gaming systems such as SEGA and Nintendo attempt to dominate the market with 
football, basketball, baseball, and hockey games that have players who look and perform 
like their real world counterparts. What was considered “high” fidelity gaming action in 
the seventies and eighties is now considered low-end computer graphics. 

The closer to reality we approach, the less forgiving the user. The user trusts the 
model to such a degree that any inconsistencies he encounters may cause him to lose 
confidence in the model representation. Most people understand that a map will have 
inconsistencies, especially in the area of vegetation. When these same discrepancies 
appear in a VE, participants seem less willing to accept the shortcomings as minor 
limitations. People forget that a VE is a tool. Instead they often become flustered with 
and untrusting of the model. 

At some point, as a model mimics reality, close is not good enough and the VE 
must be near 100% accurate. However, the point where this occurs is unknown. The 
question is, at what level of fidelity do people stop accepting model shortcomings and 
begin demanding conplete assurance? We may even be to the point where people are 
more willing to accept a simple model of elevation data covered with an aerial photo over 
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a model that depicts structures and vegetation which may be realistic but, not completely 
accurate in placement or appearance for the purpose of navigation. 

4. Iterations vs Time Limit 

Often participants and soldiers comment that “if I could only do it again, I know I 
would do it better.” They may be correct. Performance as a function of time may be so 
variable as to be statistically meaningless. Performance may depend more on the number 
of times the individual is able to maneuver through the environment. Participants in this 
experiment who were able to explore the environment looking for the control points and 
then make an additional clean run through the course outperformed those participants 
who were able to go through the course only once (Table 5.1). However, no statistical 
significance could be shown due to the limited number of participants in the group. 



Test 


1 Run VE 
Group Means 


2 Run VE 
Group Means 


Participant 
Pool Mean 
Score 


Landmark Score {Higher score is better) 


4.333 


8.000 


6.709 


Total Errors 


5.667 


6.000 


5.133 


Normalized Map Check Score 


4.127 


1.570 


2.455 


Normalized Error Score 


183.833 


54.210 


98.355 


Average Wheel Test Angular Deviation 


30.720 


19.335 


26.767 


Average Whiteboard Angular Deviation 


23.589 


18.056 


23.720 



Table 5.1. VE Participant Results 



This could be due to the opportunity to correct deficiencies in the mental map 
created as the participant explored the environment on his first trip through the course. 
To determine the validity of this hypothesis, participants could be placed into three 
similar groupings of map, real world & map, and VE & map. Instead of being giving an 
hour, ninety minutes, or two hours to study the environment and plan their route, 
participants would be allowed to go through the environment twice. The first time 
through the environment, the participant would be allowed to explore and plan his route. 
The second time through the environment, the map participant would describe his route 
without the use of a map. A real world participant would walk his planned route using 
his map and a compass. The VE participant would execute his route in the VE using his 
map. Participants would have to navigate or explain their routes with no more than one 
major or two minor errors before being allowed to move onto the evaluation phase. 
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Participants could then be evaluated on the number of repetitions it took for them to pass 
the training phase and the number of errors they committed during the evaluation phase. 

Participants in this experiment were also forced to bundle planning and rehearsal 
into one phase. In order to distinguish between the two, participants should be given ten 
to fifteen minutes to plan a preliminary route through the environment prior to exposure 
to the VE or actual environment. During the training phase participants should be 
allowed to make changes to their initial plan. This will force participants to focus their 
efforts on planning before they conduct training. 

5. Designated Route vs Participant Planned Route 

It has been shown that participants can gain path knowledge of man-made 
environments through the use of VEs [WITM 95]. In this experiment it has been 
demonstrated that less path knowledge (Chapter IV, Section A.3) and survey knowledge 
(Chapter IV, Section A.4) is obtained through exposure to a VE than through the use of 
map study of a high resolution 1,5000 orienteering map. 

What should also be studied is the performance of participants provided with a 
pre-planned route with the performance of participants who have to develop their own 
route. Limiting the task to studying and exploring a predesignated route would reduce 
the workload on the participants. This would provide them with more time to concentrate 
on the task of navigating through the environment instead of planning and navigating. If 
VEs can impart enhanced route knowledge, they could be useful in mission rehearsal of 
specific routes. 

6. Male vs Female 

Past research has suggested that males have better spatial ability and visualization 
than females [ANAS 63] [MACC 74] [LLOY 76] and may be based on genetics 
[TAVR 77]. More recent research indicates that differences in spatial ability based on 
gender are becoming less distinguishable [STUM 89] [WEST 98]. No research has been 
conducted to see if this is true for natural environments. 

In this experiment, the VE group had one female participant. On average, her 
scores were better than the mean scores for the VE participants and nearly the same as the 
mean for the entire participant pool (Table 5.2). The female participant had an 
outstanding Normalized Error Score due to her ability to quickly identify her errors and 
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construct viable solutions to resolve her situation. Her Whiteboard results indicate a 
relatively poor exocentric view of the environment. Analysis of her image (Appendix-N, 
Figure N.76) shows a shift of the control points to the west and south of their correct 
position. The remainder of her scores were well within one standard deviation of the 



means for both the VE participants and the entire pool. 



Test 


Female 

Participant 


VE Group 
Mean Score 


Participant 
Pool Mean 
Score 


Landmark Score {Higher score is better) 


6.333 


5.798 


6.709 


Total Errors 


6.000 


5.800 


5.133 


Normalized Map Check Score 


2.333 


3.104 


2.455 


Normalized Error Score 


37.930 


131.984 


98.355 


Average Wheel Test Angular Deviation 


26.833 


26.166 


26.767 


Average Whiteboard Angular Deviation 


28.154 


21.376 


23.720 



Table 5.2. Female Participant Results 



With only one female in the study, no definitive conclusions can be drawn from her 
performance. However, it brings up the interesting question if gender specific navigation 
performance is affected by the use of VEs. 

7. Colorblind vs Non Colorblind 

Past research has neglected the influence of colorblindness on navigation 
performance in man-made or natural environments. Since navigation is usually a very 
visual process, this is a majoi* shortfall. In this experiment, there were two colorblind 
participants, both in the Map Group (M2 and M4). On average their average scores were 
better then the mean scores for the Map participants and the mean for the entire 



participant pool (Table 5.3). 



Test 


Colorblind 

Participants 


Map Group 
Mean Score 


Participant 
Pool Mean 
Score 


Landmark Score {Higher score is better) 


7.665 


7.398 


6.709 


Total Errors 


5.000 


4.800 


5.133 


Normalized Map Check Score 


.690 


1.424 


2.455 


Normalized Error Score 


63.125 


83.036 


98.355 


Average Wheel Test Angular Deviation 


24.420 


26.634 


26.767 


Average Whiteboard Angular Deviation 


20.288 


22.227 


23.720 



Table 5.3. Colorblind Participant Results 
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With only two colorblind participants in the study, no conclusions can be drawn from 
their performance. However, since most VEs make more demands on the visual senses 
than any of the other senses, it brings up the interesting question of color sensitivity and 
the use of VEs to enhance navigation performance. 

8. Experience Level vs Mental Map Development 

In this and in MAJ Banker’s experiments, much emphasis was placed on the 
experience of the participant. Little research was focused on analytical ability. The 
assumption was made that individuals experienced with orienteering or military land 
navigation would best be suited as participants for this type of land navigation 
experiment. 

One pilot participant for this experiment had less than one week of military 
navigation training and no experience with orienteering. The participant had the lowest 
score on the Map Reading Test (70% correct). He scored the lowest on the Santa Barbara 
Sense-of-Direction Scale (73) and classified himself as a beginner on the Self-Ability 
Evaluation. He did score above the national average on the Guilford-Zimmerman 
Aptitude Survey (26.75) and is a Rhode Scr lar applicant. On average, his scores were 
better than the mean scores for the VE participants and the entire participant pool for 
landmark and route knowledge scores but were much worse for the survey knowledge 
tasks (Table 5.4). 



Test 


Novice Pilot 
Participant 


VE Group 
Mean Score 


Participant 
Pool Mean 
Score 


Landmark Score {Higher score is better) 


9.000 


5.798 


6.709 


Total Errors 


6.000 


5.800 


5.133 


Normalized Map Check Score 


1.111 


3.104 


2.455 


Normalized Error Score 


35.556 


131.984 


98.355 


Average Wheel Test Angular Deviation 


76.167 


26.166 


26.767 


Average Whiteboard Angular Deviation 


30.858 


21.376 


23.720 



Table 5.4. Novice Pilot Participant Results 



As stated in Chapter 4.B.2, there is significant correlation between performance 
and the Guilford-Zimmerman Aptitude Survey. Although the inexperienced novice 
participant showed difficulty with the survey knowledge tasks, he performed very well on 
the landmark and route tasks. This indicates there may be a correlation between 



114 



navigation performance and an individual’s analytical abilities, which is independent of 
experience or training. An individual’s analytical ability may impact on the way they 
organize thoughts and create mental maps. This correlation may lead to the conclusion 
that certain individuals are more prone to being natural land navigators. 

9. Medium vs Time of Exposure 

Many military missions and most hostage situations are made more complex by 
time constraints. Often individuals and teams have only days or hours to prepare for 
complex scenarios which require detailed precision. These time lines Umit resources 
which can be made available to assist in mission preparations. Research indicates that 
with limited time and exposure, VEs provide very limited performance enhancement and 
may even be counter productive [GOER 98]. 

Due to limited resources and the conplexity of the research, many experiments 
involving the usefulness of VEs to assist in the transfer of spatial knowledge to real world 
environments have been limited to exposure durations of less than an hour; often only a 
few minutes. Under these limited exposure times, participants may not be able to resolve 
the differences between the map and the VE or build an uncluttered and continuous 
mental map of the real world. Depending on the size and complexity of the model and 
actual environment, participants may need more exposure time to resolve differences and 
build a valid mental map of the terrain. 

In view of the limited amount of information on a map, the time to resolve 
differences is less than when developing a mental model from a VE. However, a map 
cannot display as much information as a VE nor can it resolve errors that may occur from 
false readings of terrain features, such as mistaking a map depiction of a depression as a 
hill. A map can provide a rough sketch of the environment with simple references that 
can easily be confirmed or refuted by the participant as he moves through the 
environment. If time is limited, a map may give an individual the rough geocentric view 
of the world needed to make simple movements through the terrain. This may be why 
the map only group outperforms the VE and real world groups for limited exposure 
times. The amount of information which can be gleaned from a map is finite and 
influences the limited level of performance one can expect for a first time pass through 
the actual environment. 
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Army doctrine recognizes that terrain is not neutral. The terrain provides a 
distinct advantage to the side which recognizes its limitations and advantages and uses 
this knowledge in the planning and conduct of operations [FMIO 93]. In the past, this 
knowledge is gained through constant exposure to the environment that provided the 
owner of the terrain with additional information which could only be learned from on 
sight observations. Depending on the individual, the acquisition of such .-mowledge takes 
days, weeks, months, or even years. With limited exposure, individuals may only gain 
route knowledge of their environment and may never gain survey knowledge depending 
on how much of the terrain they are exposed to. 

A virtual environment has many of the same characteristics as the real world with 
the added advantage of being able to look at the environment in many ways that are 
physically impossible in the actual world. Users can view the terrain from any altitude 
and position which allows them to freely move from ego to excocetric views. However, 
a virtual world cannot represent all the actual trees, holes, rocks, and other extraneous 
items in the environment. Geometric models may not be able to keep up with changes in 
the environment such as new ditches, damaged or modified structures, or fallen trees, due 
to insufficient intelligence reports, last minute corrections, or hardware and software 
limitations. This prevents the virtual world from being conpletely accurate in its 
representation of an environment. 

Taking into account the strengths and weaknesses of the media, a chart has been 
developed depicting a possible correlation between land navigation performance and 
exposure to the media (Figure 5.1). The lower left comer represents the expected 
performance of an individual with no prior knowledge of the environment, no 
navigational experience, and no navigational aids who is inserted into the environment 
and told to move from Point A to Point B. The upper right comer represents the optimal 
performance expected of an individual who has been exposed to the environment for an 
extended period of time, possibly years, who is asked to move from Point A to Point B 
with no navigational aids. 
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The graph indicates an initial superior performance by those who are exposed to a 
map of the environment before insertion. This curve gives way to the high fidelity VE 
with map curve. The low fidelity VE with map curve initially outperforms the high- 
fidelity VE due to the additional time required by the high fidelity VE user to resolve 
differences and differentiate clutter from actual valid terrain features and landmarks. 
However, once these difference are resolved, the high fidelity VE outperforms the low 
fidelity VE curve due to the additional information it can provide. The real world curve 
initially outperforms the VE curves because less time is lost resolving differences in 
mental maps. The VE curves soon pass the real world curves due to the ability to 
compress training times in a VE and to gain both egocentric and excocentric views of the 
environment using the VE. The real world curve plateaus as the user becomes roughly 
familiar with the environment but still has not transitioned into a state of survey 
knowledge of his environment. Once an individual makes the transformation to survey 
knowledge in the real world, the real world curve begins to climb and soon surpasses the 
VE curves in performance. The optimal expected performance of the real world over the 
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VE is credited to the VE’s inability to represent all the aspects of the real world. Most 
research in this area has been to the far left side of the time line. Further research is 
needed to validate the graph and curves to determine what the general values would be 
for the time scale and expected general navigational performance levels. 

10. Navigation from Sea to Shore 

Research has been conducted to identify participant abilities to navigate an open 
water virtual environment [DARK 95], a non real-time natural virtual environment 
[BANK 97], an overland hehcopter virtual environment [SULL 98], and a real time 
natural virtual environment. However, no research has been conducted to test the validity 
of using a VE to train individuals to conduct from the sea navigation. From the sea 
navigation consists of approaching land from the open ocean or sea and conducting a 
mission on or over the land. This type of navigation requires the individual to transition 
from navigating open waterways to conducting appropriate navigation over land (flying) 
or on land (driving or walking). One of the most difficult tasks in this type of navigation 
is properly identifying terrain features while on the open water to use as guides for 
transition to navigation on land. 

This type of navigation is routinely done by sailors and fishermen coming into 
ports or beach heads, Navy helicopter pilots conducting rescue missions of downed 
pilots, and by Navy Seals and Special Forces Teams who are infiltrating enemy territory 
from the sea. Reducing the chances of misinterpreting a proper land site or transition 
point can reduce parallel navigational errors, reduce mission execution times, and 
increase the chance for verall mission success. 

11. Mandatory Map Checks at Each Control Point 

One strategy used to help overcome short-term memory issues was to make a map 
check at each Control Point. One participant used this technique, M3. By conducting 
these checks, the participant verified his position within the world and was able to 
quickly memorize the route to the next control point. 

Another issue is the reason people make map checks. Map checks can be broken 
down into two basic categories, maintenance checks and recover checks (Appendix Q, 
Figure Q.l). Maintenance checks are conducted to verify or confirm an individual’s 
position, orientation, or route. A recover check is performed to determine an individual’s 
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position, orientation, and plan a new route to return to the desired location. The 
difference for why an individual makes a map check drastically impacts on the amount 
and usefulness of information they extract during the check. Individuals, who are 
completely bewildered by the environment, can look at a map for minutes without 
resolving any of the issues and can confuse themselves even more. While individuals 
who know exactly where they are in the world and on the map can quickly verify their 
planned route, make modifications, and resolve any discrepancies in their mental maps. 

Future studies should classify the type of checks an individual is performing to 
better understand how much the individual actual understands the terrain. The short term 
memory issue can be resolved by requiring each participant to conduct a fifteen second 
map check at each Control Point. Care must be taken to ensure the mandatory map 
checks do not turn the experiment into a map reading exercise. The question of map 
check classification can be resolved by asking the individual to tell the experiment 
monitor why they are making a map check. If participants indicate that they are verifying 
their location or route, the check is classified as a maintenance check. If participants 
indicate they are lost, trying to fix their position or planning a new route, then they are 
performing a recover check. 

12. Orienteering Map vs 1:50,000 Map 

Previously we discussed how the fidelity of the map may have played a major role 
in the performance of the Map Group Participants (Chapter 4.B.9). It is true that most 
military navigation is performed on a 1:50,000 (Appendix F, Figure F.l) or 1:24,(X)0 
(Appendix F, Figure F.2) map with rough sketches, blue prints, or aerial photos of the 
target areas. Most military personnel would never be afforded such a high resolution 
map depicting most linear features and categorizing the terrain to the degree that the 
experiment’s 1:5,(XX) map provided. A more realistic study may use the 1:24,000 map in 
conjunction with an aerial photo and sketches of the control point areas instead of the 
high fidelity orienteering map. Participants could then draw their proposed route on an 
8.5x1 1 inch aerial photo of the terrain. This would provide research monitors with the 
same resolution map to track and evaluate participant movement without providing a map 
with such a high degree of fidelity. This could also help to reduce problems with 
resolving distances for participants who are accustomed with standard military map 
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distances. Figure 5.2 shows the potential performance levels of individuals using 




Real Time VEs 

Non Real Time VEs 

1:5,000 Map 

1:24,000 Map 

1:50,000 Map 



Figure 5.2. Medium Resolution vs Performance Diagram 
differing training media. The performance curves for real time VEs, non real time VEs, 
and 1:5,000 scale map participants at the one-hour exposure mark are based on the results 
of the Banker experiment [BANK 97] and this thesis. The 1:24,000 and 1:50,000 scale 
map curves at the one-hour mark are predicted results. The optimal exposure times in 
each medium before performance levels off have not been verified through research. 
These times will vary based on the conq>lexity of the environment and the abilities of the 
individual navigators. 

13. Run Route Backwardls 

To reduce the memorization requirements and the length of the course, 
participants could be asked to plan and study a route that had only five or six controls. 
To increase the difficulty in the execution of the course while reducing the distance to be 
covered, the participants could then be asked to run the course in reverse order. For 
example, participants would be asked to plan a route through the control points in 
alphabetical order (A, B, C, D, E). When the participants are taken to the course, they are 
then informed they will run the course in reverse order (E, D, C, B, A). 
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This will allow participants in the VE and Real World Groups to have plenty of 
time to con^)lete the course one or more times. It will also test to see if participants can 
play their mental movies or traverse their list of mental directions in reverse order. This 
would help to indicate that survey knowledge is being gleaned from the training phase 
and not merely route knowledge. 

14. Introduction of Secondary Task 

For most military missions, navigation is not the primary task. Instead it is a 
secondary task to get an individual or group of individuals to a location so that the 
primary task can be accomplished. Furthermore, during most military navigation 
operations, the task shares focus with the need to provide security and communicate with 
other entities. During this experiment, participants were asked to concentrate on 
navigating through the course as their primary task. At no time was an individual 
required to conduct more than one-task simultaneously during the execution phase of the 
experiment. 

To provide more validity to the experiment, it would be prudent to quantitatively 
show that after training in a VE, individual navigation performance improved in the real 
world environment while secondary tasks are being performed. The secondary task 
would need to be simplistic in design to ensure additional training is not required to 
perform the task. At the same time, it must be complicated enough to require mental 
resources to be concentrated on the task for its successful con^letion. 

15. Compare Execution Through VE and Real World 

To better determine if a virtual world can substitute for its real world counterpart, 
an experiment testing navigation performance in a virtual world compared to navigation 
performance in the real world may provide some further insight. Both groups would be 
provided with the same training materials to plan a route through a course. Next, the 
participants would execute their planned routes through the actual navigation course or 
through a VE of the course. Measurements could then be taken on errors, distance per 
error, map checks. Wheel Tests, and Whiteboard Tests to determine if the two groups 
perform at relatively similar levels. This would help to ascertain if VEs are a viable 
substitute to a full-scale mock-up of the actual terrain. 
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APPENDIX A. EXPERIMENT OUTLINE 



1) In Brief/Consent Form 

a) Time - 5 Min 

b) Location - CS Student Conf Room 

c) OIC - CPT Simon R. Goerger 

d) Materials - Consent Form, Privacy Act Statement, Minimal Risk Consent Form, 
Subject Roster, pencils, Fort Ord Map (confirm the subject has not been on the 
course terrain before), In Briefing Script 

2) Color Blindness Test/Self Evaluation Questionnaires/Map Reading Test 

a) Time -15 Min 

b) Location - CS Student Conf Room 

c) OIC - CPT Simon R. Goerger 

d) Materials - Color Charts (1 min), Self AbUity Evaluation Sheet (1 min), Santa 
Barbara Sense of Direction Scale Questionnaire (3 min). Map Reading Test (5 
min), pencil 

e) Grading (5 min) 

3) Spatial Orientation 

a) Time -15 Min 

b) Location - CS Student Conf Room 

c) OIC - CPT Simon R. Goerger 

d) Materials - Guilford-2Iimmerman Aptitude Tests (10 min), pencils, answer sheets, 

e) Grading and Grouping (5 min) 

f) Groups 

i) Group A - Upper 50 percentile 

ii) Group B - Lower 50 percentile 

4) Interface Familiarization (VE Only) 

a) Time - 15 Min minimum 

b) Location - Graphics Lab 

c) OIC - CPT Simon R. Goerger 

d) Materials -SGI machine. Performer Town Model, Flybox instructions. Virtual 
Environment Briefing Script, Interface Familiarization Checklist 

e) Movement (15 min minimum) 
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5) Training 

a) Map Group 

(1) Time - 60 Min 

(2) location - CS Student Conf Room 

(3) OIC - CPT Simon R. Goerger 

(4) Materials - Fort Ord Orienteering Map, Participant Task List, Map 
Marking Instructions, red alcohol marker, alcohol marker eraser, pencil, 
scratch paper, orienteering clue sheet. Map Group Briefing Script, Training 
Evaluation Sheet 

b) Real World Group 

(1) Time -60 Min 

(2) Location - Fort Ord Orienteering Course 

(3) OIC - CPT Simon R. Goerger 

(4) Materials - Fort Ord Orienteering Course, Fort Ord Orienteering Map, 
Participant Task List, Map Marking Instructions, red alcohol marker, 
alcohol marker eraser, pencil, scratch paper, orienteering clue sheet, 
compass. Real World Group Briefing Script, Training Evaluation Sheet 

c) Virtual Enviromnent Group 

(1) Time -60 Min 

(2) Location - Graphics Lab CPT Simon R. Goerger 

(3) OIC - CPT Simon R. Goerger 

(4) Materials - Elvis (SGI ) w/ flybox and 2rV40” screen configuration or 
projector. Fort Ord Model, Fort Ord Orienteering Map, Participant Task 
List, Map Marking Instructions, Flybox instructions, red alcohol marker, 
alcohol marker eraser, pencil, scratch paper, orienteering clue sheet. 
Virtual Environment Briefing Script, Training Evaluation Sheet 

6) Testing (est Time 120 Minutes - travel to Fort Ord Orienteering Course, run the 
course, and return). 

a) Time - Travel Time 30 Min (total); Run Course 90 Min; Total Time (120 min) 

b) Location - Fort Ord Orienteering Course 

c) OIC - CPT Simon R. Goerger 
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d) Materials - Clipboard with subject’s map & designated route, corrpass. Think Out 
Loud Instructions, Data Collection Sheet, red pen to record data, blue alcohol pen, 
stop watch/timer. Color Wheel for Tasks 3.1. & 5.1, White Board with ten 
magnets, rucksack frame w/GPS system, helmet & camera, water, first aid kit 
(cellular phone). Course Briefing Script, blind fold (for movement to course), 
spare clue sheet & color wheel arrows, Tecnu (for poison oak) 

e) Tasks: 

i) Task 1. {Path Knowledge) Move from the starting point to Checkpoint #1 

along designated route, (measure elapsed time and number of errors; mark 
deviation from route on map) 

ii) Task 2. {Path Knowledge) Move from Checkpoint #1 to Checkpoint #2 along 

designated route, (measure elapsed time and number of errors; mark 
deviation from route on map) 

iii) Task 3.1. {Survey Knowledge) Take bearings to SP, CP #5, and CP #9 at the 

south side of CP #4) 

iv) Task 3.2. {Path Knowledge) Move from Checkpoint #2 to Checkpoint #3 

along designated route, (measure elapsed time and number of errors; mark 
deviation from route on map) 

v) Task 4. {Path Knowledge) Move from Checkpoint #3 to Checkpoint #4 along 

designated route, (measure elapsed time and number of errors; mark 
deviation from route on map) 

vi) Task 5.1. {Survey Knowledge) Take bearings to CP #1, CP #6, and CP #8 at 

the south side of CP #4) 

vii) Task 5.2. {Path Knowledge) Move from Checkpoint #4 to Checkpoint #5 

along designated route, (measure elapsed time and number of errors; mark 
deviation from route on map) 

viii) Task6. {Path Knowledge) Move from Checkpoint #5 to Checkpoint #6 

along designated route, (measure elapsed time and number of errors; mark 
deviation from route on map) 

ix) Task 7. {Path Knowledge) Move from Checkpoint #6 to Checkpoint #7 along 

designated route, (measure elapsed time and number of errors; mark 
deviation from route on map) 

x) Task 8. {Path Knowledge) Move from Checlq)oint #7 to Checkpoint #8 along 

designated route, (measure elapsed time and number of errors; mark 
deviation from route on map) 

xi) Task 9. {Path Knowledge) Move from Checkpoint #8 to Checkpoint #9 along 

designated route, (measure elapsed time and number of errors; mark 
deviation from route on map) 

xii) Task 10. {Survey Knowledge) Have subject indicate bearing and route he 

must traverse to make it to Checkpoint #4. Have subject return to 
Checkpoint #4. (mark route and any turn which leads the subject away 
from Checkpoint #4. Allow a maximum of ten minutes to return to 
Checkpoint #4) 
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xiii) Task 11. (Survey Knowledge) Have subject arrange magnets on the white 
board indicating the location of the starting point and nine checkpoints. 
Measure time and note method of magnet placement (i.e. in order of visit, 
outside-in, or inside-out). Take picture of final results (allow 5 minutes 
maximum). 

g) Error (Definition) 

Subject strays from designated route (5 meters firom designated route on a 
path/trail/road; 15 meters from cross country designated route), (record one error) 

7) Debriefing. 

a) Time - 30 Min 

b) Lx)cation - Graphics Lab 

c) OIC - CPT Simon R. Goerger 

d) Materials - Clipboard with subject’s map & designated route. Data Collection 
Sheet, red pen to record data, GPS system. Troop (PC) w/ Arcview and Fort Ord 
Maps, digital camera. Participant Questionnaire(s), Researcher’s Script 

e) Administer questionnaire(s); down load GPS datum and d ?play on aerial photo 
using Arcview. 

f) Discuss rouu . 

i) Have the subject complete the Debriefing Questionnaire. Read their answers 

and ask for any clarification. 

ii) Walk the subject through his route using the subjects planned route and the 

GPS data down loaded fi'om the Message Pad and plotted on the aerial 
photo in Arcview. 

(a) Have the subject to explain why they deviated fi*om their route at those 
locations where the two differ. 

(b) Have the subject explain when & how they determined they were off 
course. 

(c) Have the subject explain how they recovered. 

iii) Ask the subject if he would have done anything different in the training phase 

now that has completed the experiment. 

iv) How much time does the subject spend playing computer games or working 

with computer graphics (more than an hour a day, a couple hours a week, 
once or twice a month, rarely, never)? 
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APPENDIX B. TASK LISTING 



Task 1. {Path Knowledge) Move from starting point to Checkpoint #1 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 2. {Path Knowledge) Move from Checkpoint #1 to Checkpoint #2 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 3.1. {Survey Knowledge) Take bearings to SP, CP #5, and CP #9 at the 
south side of CP #4). 

Task 3.2. {Path Knowledge) Move from Checkpoint #2 to Checkpoint #3 
along designated route (measure elapsed time and number of errors; mark deviation from 
route on map). 

Task 4. {Path Knowledge) Move from Checkpoint #3 to Checkpoint #4 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 5.1. {Survey Knowledge) Take bearings to CP #1, CP #6, and CP #8 at 
the south side of CP #4). 

Task 5.2. {Path Knowledge) Move from Checkpoint #4 to Checkpoint #5 
along designated route (measure elapsed time and # errors; mark deviation from route on 
map). 

Task 6. {Path Knowledge) Move from Checkpoint #5 to Checkpoint #6 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 7. {Path Knowledge) Move from Checkpoint #6 to Checkpoint #7 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 8. {Path Knowledge) Move from Checkpoint #7 to Checkpoint #8 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 
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Task 9. {Path Knowledge) Move from Checkpoint #8 to Checkpoint #9 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 10. (Survey Knowledge) Have subject indicate bearing and route he must 
traverse to make it to Checkpoint #4. Have subject return to Checkpoint #4 (mark route 
and any turn which leads the subject away from Checkpoint #4. Allow a maximum of ten 
minutes to return to Checkpoint #4). 

Task 11. (Survey Knowledge) Have subject arrange magnets on the white 
board indicating the location of the starting point and nine checkpoints. Measure time and 
note method of magnet placement (i.e. in order of visit, outside in, or inside out). Take 
picture of final results (allow 5 minutes maximum). 
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APPENDIX C. BRIEFING SCRIPTS 



1. GENERAL 

The scripts in the appendix appear in the same format utilized for the experiment and do 
not follow the standard thesis format utilized in the chapters of this document. This appendix 
consists of five briefing scripts: In Briefing, Control Group Briefing, Map Group Briefing, 
Virtual Environment Briefing, and the Course Briefing. Each participant receives the In 
Briefing and Course Briefing. The participants are exposed to either the Control Group 
Briefing, Map Group Briefing, or Virtual Environment Briefing depending on which group they 
are assigned. This appendix also contains the Debriefing hand out. 

2. IN BRIEFING 

Welcome to the Naval Postgraduate School’s Computer Science Department. My name 

is . Thank you for your assistance with today’s experiment. Today’s experiment 

deals with dismounted navigation in natural terrain. 

This experiment is not a test of your intelligence or performance. Rather, it is an 
evaluation of navigational tools. (For Military Personnel) Your performance will not be 
recorded in your personnel records but is intended for research purposes only. All information 
collected is for academic research only. Prior to starting the experiment you will be asked to 
read and sign a series of consent forms. Upon signing the consent forms, you will take self- 
evaluation, map reading, and spatial orientation exams. After the tests, you will under go a 
sixty-minute train-up period prior to moving to the navigation course. Upon completing the 
course, you will be brought back to Spanagel Hall for a short debriefing. 

If there are no questions, please read and sign this consent form. 
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3. CONTROL GROUP BRIEFING 



In front of you is a map of an orienteering course as well as the actual terrain 
depicted on the map. You also have a clue sheet describing the location of the control 
points and photos of the control points. The map, photos, and terrain are for your use to 
study and plan the route you will be using to navigate the course. 

You have sixty minutes to study the map and terrain. Your planned route must 
navigate you through the nine checkpoints in order. (Show the participant the 
checkpoints in order then point out each checkpoint in the photo.) Beginning at the 
designated starting point, you will go to CPI, then to CP2, then to CPS, ... and finally to 
CP9. The checkpoints are described in the clue sheet provided. You may take the clue 
sheet with you when you go on the course. Before the end of the sixty-minute study 
phase, you will mark your planned route on the map using a red alcohol marker. 

After con^leting the study phase, you will be escorted back to the starting point to 
run the -oute you designated on your laminated map. While navigating the course, you 
will no: have the map nor will you be allowed to use a compass. During the execution of 
the course, you may request a thirty seconds map or compass check; or a sixty-second 
map and compass check. You can request as many map or conpass checks as you wish, 
but each check will be recorded. If you decide to deviate firom your previously planned 
route, you may request the map to mark your newly planned route. 

Do you have any questions before we begin? 
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4. MAP GROUP BRIEFING 



In front of you is a map of an orienteering course. You also have a clue sheet 
describing the location of the control points as well as photos of the control points. The 
map and photos are for your use to study and plan the route you will be using to navigate 
the course. 

You have sixty minutes to study the map. Your planned route must navigate you 
through the nine checkpoints in order. (Show the participant the checkpoints in order 
then point out each checkpoint in the photo.) Beginning at the designated starting point, 
you will go to CPI, then to CP2, then to CP3, . . . and finally to CP9. The checkpoints are 
described in the clue sheet provided. You may take the clue sheet with you when you go 
on the course. Before the end of the sixty-minute study phase, you will mark your 
planned route on the map using a red alcohol marker. 

After completing the study phase, you will be taken to the navigation course to run 
the route you designated on your laminated map. While navigating the course, you will 
not have the map nor will you be allowed to use a compass. During the execution of the 
course, you may request a thirty seconds map or compass check; or a sixty-second map 
and compass check. You can request as many map or compass checks as you wish, but 
each check will be recorded. If you decide to deviate from your previously planned route, 
you may request the map to mark your newly planned route. 

Do you have any questions before we begin? 
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5. VIRTUAL ENVIRONMENT GROUP BRIEFING 



Prior to beginning the study phase you will under go a fifteen-minute model 
familiarization phase. This is to help you become comfortable with the model controls 
prior to starting the experiment. The model you wiU be using for this phase bears no 
resemblance to the actual model to be used during the training phase. You will be 
required to show proficiency with the interface prior to moving on to the terrain model. 

In fi"ont of you are the 3-screen configuration, a joystick interface, and a list of 
instructions for the use of the interface (demo controls). Please feel fi’ee to explore the 
environment and controls for the next few minutes. When you feel confident with the 
controls, I will walk you through a serious of questions to demonstrate your expertise. 

(Conduct Familiarization Phase; after the participant demonstrates proficiency 
with the interface, load up the terrain model and begin the training phase) 

In firont of you is a map of an orienteering course as well as a high fidehty 3-D 
model of the terrain depicted on the map. You also have a clue sheet describing the 
location of the control points as well as photos and screen capture images of the control 
points. The map, photos, and VE are for your use to study and plan the route you will be 
using to navigate the course. 

You have sixty minutes to study the map and VE. Your planned route must 
navigate you through the nine checkpoints in order. (Show the participant the 
checkpoints in order then point out each checkpoint in the photo.) Beginning at the 
designated starting point, you will go to CPI, then to CP2, then to CP3, ... and finally to 
CP9. The checkpoints are described in the clue sheet provided. You may take the clue 
sheet with you when you go on the course. Before the end of the sixty-minute study 
phase, you will mark your planned route on the map using a red alcohol marker. 

After completing the training phase, you will be taken to the navigation course to 
run the route you designated on your laminated map. While navigating the course, you 
will not have the map nor will you be allowed to use a conpass. During the execution of 
the course, you may request a thirty seconds map or compass check; or a sixty-second 
map and compass check. You can request as many map or conq)ass checks as you wish, 
but each check will be recorded. If you decide to deviate firom your previously planned 
route, you may request the map to mark your newly planned route. 

Do you have any questions before we begin? 
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6. COURSE BRIEFING 



Pick-up participant from the Graphics Lab. 

Move participant to the Fort Ord orienteering course. 

Move participant to start point: 

Brief the participant on animals and ammunition 

“You are at the start point of the Navigation Course. During the experiment, I may stop 
you and ask you to answer questions. You must navigate the nine checkpoints in order. 
Each control point will be identified by a control point marker (show participant a control 
marker) which you must touch prior to moving to the next control point. Once you touch 
a control marker, I will tell you which marker it is. If it is the correct marker, I will give 
you further instructions. If it is the incorrect marker, I wiU not say anything other then the 
marker’s number. I wiU not stop you unless you attempt to cross the course boundaries 
(show participant the boundaries). You may request the compass for a thirty second 
compass check; the map for a thirty second map check; or the map and compass for a 
sixty second compass and map check. These checks wiU be recorded and timed by me. If 
you determine that you would like to change your route, you may request the map and a 
blue marker to mark changes to your proposed route. You wiU have sixty seconds to 
mark your new route. You may request an additional sixty seconds if you deem it 
necessary. You have sbcty minutes to make it as far as you can along your planned route. 
From now until conq)letion of the navigation course do not interact with anyone. Before 
you begin, do you have any questions?” 

TASK 1: START POINT TO CHECKPOINT ONE. 

Task; “Your first task is to move from the start point to checkpoint one along your 
designated route.” 

Condition: “Without a map or interaction with anyone move from start point to 

checkpoint one along your preplanned route. If you deviate from the designated route 
you will be allowed to continue your movement unless you attempt to go outside the 
course boundaries. You may deviate 5m from your route, if you are on a trail, or 15m, if 
you are conducting cross-country movement before you are assessed an error. You can 
move back and forth along your route without being assessed an error. If you deviate 
from your path for more then 15 continues minutes and are not make progress towards the 
intended control point, I will stop you, show you your location on the map, and give you 
sixty seconds to mark a new route to the appropriate control point.” 

Standard: “Do the best you can.” 

“Ready,... Begin.” 
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TASK 2: CHECKPOINT ONE TO CHECKPOINT TWO. 



Task: “Checkpoint one. Your next task is to move from the checkpoint one to 

checkpoint two along your planned route. Conditions and standards are unchanged.” 

TASK 3.1.A, B, C: SPATIAL AWARENESS TEST I. 

Stop timer 

Stop participant at spatial awareness test area. 

“Checkpoint two. Stop, I am going to have you identify the direction to three 
checkpoints.” 

Place the color wheel platform in its base on the south side of checkpoint. 

Task: “Identify the direction to the start point, checkpoint five, and checkpoint nine.” 
Show participant arrows as you state their names. 

Condition: “Given a color coded, 360-degree wheel and three arrows, identify the 

direction to the start point, checkpoint five, and checkpoint nine by placing the appropriate 
arrow in the direction of its checkpoint.” 

Standard: “Unchanged.” 

Record the time it takes the participant to perform the Wheel task and the 
orientation of the participant (looking north, south, east, rotates in the direction of the 
arrows, etc). Once done, photo graph the wheel, remove wheel platform from its stand, 
and have participant continue to checkpoint three. 

TASK 3.2: CHECKPOINT TWO TO CHECKPOINT THREE. 

Task: “Your next task is to move from the checkpoint two to checkpoint three along your 
planned route. Conditions and standards are unchanged. Ready,... Begin.” 

Start timer 

TASK 4: CHECKPOINT FOUR TO CHECKPOINT FIVE. 

Task: “Checkpoint three. Your next task is to move from the checkpoint three to 

checkpoint four along your planned route. Conditions and standards are unchanged.” 
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TASK 5.1.A, B, C: SPATIAL AWARENESS TEST I. 



Stop timer 

Stop participant at spatial awareness test area. 

“Checkpoint four. Stop, I am going to have you identify the direction to three 
checkpoints.” 

Place the color wheel platform in its base on the south side of checkpoint. 

Task; “Identify the direction to checkpoint one, checkpoint six, and checkpoint eight.” 
Show participant arrows as you state their names. 

Condition: “Given a color coded, 360-degree wheel and three arrows, identify the 

direction to checkpoints one, six, and eight by placing the appropriate arrow in the 
direction of its checkpoint.” 

Standard: “Unchanged.” 

Record the time it takes the participant to perform the Wheel task and the orientation 
of the participant (looking north, south, east, rotates in the direction of the arrows, 
etc). Once done, photo graph the wheel, remove wheel platform from its stand, and 
have participant continue to checkpoint five. 

TASK 5.2: CHECKPOINT TWO TO CHECKPOINT THREE. 

Task; “Your next task is to move from the checkpoint four to checkpoint five along your 
planned route. Conditions and standards are unchanged. Ready,... Begin.” 

Start timer 

TASK 6: CHECKPOINT FIVE TO CHECKPOINT SIX. 

Task: “Checkpoint five. Your next task is to move from the checkpoint five to 

checkpoint six along your planned route. Conditions and standards are unchanged.” 

TASK 7: CHECKPOINT SIX TO CHECKPOINT SEVEN. 

Task: “Checkpoint six. Your next task is to move from the checkpoint six to checkpoint 
seven along your planned route. Conditions and standards are unchanged.” 
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TASK 8: CHECKPOINT SEVEN TO CHECKPOINT EIGHT. 



Task: “Checkpoint seven. Your next task is to move from the checkpoint seven to 
checkpoint eight along your planned route. Conditions and standards are unchanged.” 

TASK 9: CHECKPOINT EIGHT TO CHECKPOINT NINE. 

Task: “Checkpoint eight. Your next task is to move from the checkpoint eight to 

checkpoint nine along your planned route. Conditions and standards are unchanged.” 

TASK 10.1: CHECKPOINT 4 IDENTIFICATION. 

While standing at checkpoint nine: 

Stop timer 

Task: “Checkpoint nine, finish point. Your next task is to identify the location of 

checkpoint four from where you are.” 

Condition: “Point to checkpoint four and tell me where checkpoint four is from here, 
(i.e., twenty meters and in this direction).” 

Standard: “Unchanged.” 

TASK 10.2: DESCRIBE ROUTE FROM CHECKPOINT NINE TO START 
POINT 

Task: “Your next task is to describe what you consider the easiest route you would take 
to move from here to checkpoint four.” 

Condition: “Without a map, describe the route you would take to move from checkpoint 
nine to checkpoint four.” 

Standard: “Unchanged.” 

TASK 10.3: CHECKPOINT NINE TO START POINT (if described route would 
take them in the general location of the start point) 

Task: “Your next task is to move from checkpoint nine to checkpoint four using the route 
you just described.” 

Condition: “Again, do not interact with anyone to include the researcher. You may not 
request a map or a compass check.” 

Standard: “You have ten minutes, otherwise standards are unchanged.” 
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“Ready,... Begin” 



Start timer 

Reach checkpoint #4 or ten minutes has elapsed. 

FINISH 

Stop timer 

“Stop. Congratulations you have completed the navigation portion of this experiment. 
We will now return to the vehicle for one final test before returning to the laboratory.” 

TASK 11: WHITE BOARD TEST. 

Task: “Your final task is to create a top down representation of the start point and nine 
control points.” 

Condition: “Without a map or interaction with anyone take the ten magnets labeled with 
the start point and nine checkpoints (show the participant the magnets) and place them 
on a clean white board in proper perspective to each other. You are attempting to create 
a top down view of the checkpoints, actual distance between points does not matter, 
however, relative locations to each checkpoint does. Until you feel you are finished or 
five minutes has elapsed, you may place and move the magnets as you wish. 

Standard: “Do the best you can.” 

“Any questions,... Ready,... Begin.” 

Start timer 

Stop the timer when the participant indicates he has finished or ten minutes has 
elapsed, which ever occurs first. Observe the participant and note his method for 
placing the magnets (Le. in order of visit, outside in, or inside out). Take a picture of 
the final results (allow participant 5 minutes maximum to petform the task). 



Stop timer 

“Stop. Congratulations on completing the final task for this experiment. We will now 
return to NFS for a final debriefing session.” 

Move participant back to the Graphics Lab for debriefing. 
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7. DEBRIEFING 



The use of virtual environments in training and education has been an expanding 
field for the last two decades. With recent developments in computer systems, virtual 
reality models are now able to display much higher fidelity. In order to insure we are 
providing a positive training transfer and properly replicating real world environments, 
research is being conducted in the levels of detail required in models. 

The study you have just completed is concerned with gathering information on 
how individuals navigate through conq)lex virtual environments. You spent a session 
planning and studying a route demonstrating route knowledge. FinaUy, you demonstrated 
spatial knowledge of the terrain through estimating bearings to known points and 
movement to an unplanned location. 

Three separate groups were examined in order to determine performance levels. 
All three groups were given an orienteering map on which they designated their routes 
prior to running the navigation course. The first group was only allowed to study the map 
for 60 minutes. The second group was given the map and allowed to move through the 
terrain for 60 minutes prior to running the course. The third group was give the map and 
was allowed to maneuver through a real time, high fidelity virtual representation of the 
terrain for 60 minutes. 

The research personnel observed and recorded information based on the 
experience and behavior of the participants in order to gather the information equipped for 
the redesign and implementation of a more useful virtual model. The notes and 
observations collected will be used for the purpose of establishing standards for model 
development. 

Your assistance in this project will contribute to the production of more useful 
virtual environments that provide users with spatial knowledge and better navigational 
skills. With the information gathered from your experience and the experience of other 
participants, we are discovering what people generally use as navigational cues in the 
virtual and real world environments. This information will assist in the design of future 
virtual reality models that will be adaptive to a variety of individual needs. 

If you have any questions about the study, please ask your research assistant. 
Until 30 July 1998. please do not discuss this experiment with anyone except our 
research personnel to prevent influencing any future participants . Thank you for your 
participation in this study. 

The research supervisor, CPT Simon R. Goerger, for this study can be contacted at 
(408) 656 - 4077 or Email: srgoerge@cs.nps.navy.mil. 
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APPENDIX D. CONSENT FORMS 



1. GENERAL 

The forms in the appendix appear in the same format utilized for the experiment and do 
not follow the standard thesis format utilized in the chapters of this document. This appendix 
consists of three documents: Consent Form, Minimal Risk Consent Statement, and the Privacy 
Act Statement. Each participant is required to read and sign these documents before he is 
allowed to participate in the study. A research monitor observes and verifies the signing of 
each document. The format and content of these documents is based on the forms used in 
MAJ William Banker’s land navigation experiment [BANK 97]. 
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2. CONSENT FORM 



PARTICIPANT CONSENT FORM 

1. Introduction. You are invited to participate in a study of spatial awareness of natural 
and virtual environments. With information gathered from you and other participants, 
we hope to discover insight on navigational aids used to move through virtual 
environments during dismounted navigation of natural terrain. We ask you to read and 
sign this form indicating that you agree to be in the study. Please ask any questions you 
may have before signing. 

2. Background Information. The Naval Postgraduate School NPSNET Research Group 
is conducting this study. 

3. Procedures. If you agree to participate in this study, the researcher will explain the 
tasks in detail. There will be two sessions: a) 30 pretest phase and 2) training and 
execution phases lasting approximately five hours in duration, during which you will be 
expected to accomplish a number of tasks related to navigating natural terrain. 

4. Risks and Benefits. This research involves no risks or discomforts greater then those 
encountered in ordinary hike through rolling, wooded terrain. The benefits to the 
participants are gaining techniques for enhancing spatial knowledge of unfamiliar 
environments and contributing to current research in human-computer interaction. 

5. Compensation. No tangible reward will be given. A copy of the results will be 
available to you at the conclusion of the experiment. 

6. Confidentiality. The records of this study will be kept confidential. No information 
will be publicly accessible which will possibly identify you as a participant. 

7. Voluntary Nature of the Study. If you agree to participate, you are free to withdraw 
from the study at any time without prejudice. You will be provided a copy of this form 
for your records. 

8. Points of Contact. If you have any further questions or comments after the completion 
of the study, you may contact the research supervisor, CPT Simon R. Goerger, at 
(408) 656 - 4077 (Email: srgoerge@cs.nps.navy.mil). 

9. Statement of Consent. I have read the above information. I have asked all question 
and have had my questions answered. I agree to participate in this study. 



Participant’s Signature 



Date 



Researcher’s Signature 



Date 



3. MINIMAL RISK CONSENT STATEMENT 



NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
MINIMAL RISK CONSENT STATEMENT 

Participant: VOLUNTARY CONSENT TO BE A RESEARCH PARTICIPANT IN: Virtual 

Environments and Navigation in Natural Environments 

1. I have read, understand and been provided "Information for Participants" that provides the 
details of the below acknowledgments. 

2. I understand that this project involves research. An explanation of the purposes of the 
research, a description of procedures to be used, identification of experimental procedures, 
and the extended duration of my participation have been provided to me. 

3. I understand that this project does not involve more than minimal risk. I have been informed 
of any reasonably foreseeable risks or discomforts to me. 

4. I have been informed of any benefits to me or to others that may reasonably be expected from 
the research. 

5. I have signed a statement describing the extent to which confidentiality of records identifying 
me will be maintained. 

6. I have been informed of any coirq^ensation and/or medical treatments available if injury 
occurs and is so, what they consist of, or where further information may be obtained. 

7. I understand that my participation in this project is voluntary, refusal to participate will 
involve no penalty or loss of benefits to which I am otherwise entitled. I also understand that 
I may discontinue participation at any time without penalty or loss of benefits to which I am 
otherwise entitled. 

8. I understand that the individual to contact should I need answers to pertinent questions about 
the research is Rudy Darken, Ph.D., Principal Investigator, and about my rights as a research 
participant or concerning a research related injury is the Modeling Virtual Environments and 
Simulations Chairman. A full and responsive discussion of the elements of this project and 
my consent has taken place. 

Medical Monitor: Flight Surgeon, Naval Postgraduate School 



Signature of Principal Investigator 


Date 


Signature of Volunteer 


Date 


Signature of Witness 


Date 
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4. PRIVACY ACT STATMENT 



NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
PRIVACY ACT STATEMENT 

1. Authority: Naval Instruction 

2. Purpose: S patial Cognition information will be collected to enhance knowledge, or 
to develop tests, procedures, and equipment to improve the development of Virtual 
Environments. 

3. Use: Spatial Cognition information will be used for statistical analysis by the 
Departments of the Navy and Defense, and other U.S. Government agencies, 
provided this use is compatible with the purpose for which the information was 
collected. Use of the information may be granted to legitimate non-government 
agencies or individuals by the Naval Postgraduate School in accordance with the 
provisions of the Freedom of Information Act. 

4. Disclosure/Confidentiality: 

a. I have been assured that my privacy will be safeguarded. I will be assigned a 
control or code number which thereafter will be the only identifying entry on 
any of the research records. The Principal Investigator will maintain the cross- 
reference between name and control number. It will be decoded only when 
beneficial to me or if some circumstances, which is not apparent at this time, 
would make it clear that decoding would enhance the value of the research data. 
In all cases, the provisions of the Privacy Act Statement will be honored. 

b. I understand that a record of the information contained in this Consent Statement 
or derived fi'om the experiment described herein will be retained permanently at 
the Naval Postgraduate School or by higher authority. I voluntarily agree to its 
disclosure to agencies or individuals indicated in paragraph 3 and I have been 
informed that failure to agree to such disclosure may negate the purpose for 
which the experiment was conducted. 

c. I also understand that disclosure of the requested information, including my 
Social Security Number, is voluntary. 



Signature of Volunteer Name, Grade/Rank (if applicable) DOB SSN Date 



Signature of Witness 



Date 



APPENDIX E. QUESTIONNAIRES AND TESTS 

1. GENERAL 

The items in the appendix appear in the same format utilized for the experiment 
and thus do not follow the standard thesis format utilized in the chapters of this 
document. This appendix consists of eight documents: Land Navigation Questionnaire, 
Self Ability Evaluation, Santa Barbara Sense-of-Direction Scale, Map Reading Test, 
Guilford-Zimmerman Aptitude Survey, Practice Model Test, and two Debriefing 
Questionnaires. 

The Land Navigation Questionnaire (Appendix E.2) provides a very general 
background of the participant. The participant, prior to arriving to the experiment site, 
completes this questionnaire. 

The Self Ability Evaluation (Appendix E.3) is a qualitative self analysis of an 
individual’s navigational ability. It provides a participant with general limits from which 
to appraise his perceived navigation aptitude. The left end of the scale is valued at 0.00 
and the right end of the bar line is valued at 1.00. Values measured from 0.00 to 0.33 are 
assessed as beginning navigators. From 0.33 to 0.66 is ranked as an intermediate 
navigator. Values of 0.66 to 1.00 are evaluated as experts. 

The Santa Barbara Sense-of-Direction Scale (Appendix E.4) is a quantitative self- 
evaluation of navigational ability. The University of California at Santa Barbara 
developed the scale. An individual’s score is calculated by reversing the values of 
questions 2, 6, 8, 10, 11, 12, 13, and 15. For example, is the participant answered 
question number two as “3”, the question is given a numerical value of “5”. Once the 
values for the above questions are reversed, sum the value of each question and divide the 
total by the number of questions answered. The lower the resulting score the more 
confidant an individual is in their navigational abilities. The University of California at 
Santa Barbara calculates scale’s mean score of 3.54 with a standard deviation of 1.03. 
For this experiment, the mean score was 2.62 with a standard deviation of 0.57. 

The Map Reading Test (Appendix E.5) is comprised of twenty questions dealing 
with terrain feature identification. The test is designed to determine if an individual can 
read the terrain features on a map and associate them to real world terrain features. The 
first fifteen questions relate to properly naming terrain features from 1:50,000 scale 



143 



military maps and an orienteering map. The last five questions dealt with associating 
images of terrain features to map depictions of terrain features. The answers for the test 
are listed in Table E.l. Each question is worth one point. If a participant misidentifies a 
linear terrain feature they receive 0.5 points for the question. For example if the terrain 
feature is a stream and the participant classifies it as a road, they receive 0.5 points for the 
question. However, if the participant describes a stream as a draw, they receive no credit. 
Participants must score 65% (13 out of 20) or better to be allowed to participate in the 
study. Scores ranged from 13 to 20 with a mean score of 87.5%. 



Question 


Answer 


Feature 


1.1 


B 


Draw 


1.2 


I 


Spur/Finger 


1.3 


H 


Saddle 


1.4 


A 


Depression 


1.5 


C 


Hill Top 


2.1 


F 


Road/Trail 


2.2 


B 


Draw 


2.3 


E 


Ridge Line 


2.4 


L 


Valley 


2.5 


I 


Spur/Finger 


3.1 


I 


Spur/Finger 


3.2 


C 


Hill Top 


3.3 


G 


Road/Trail Intersection 


3.4 


J 


Stream/River 


3.5 


B 


Draw 


4.1 


F 


Road/Trail 


4.2 


D 


Hill Top 


4.3 


A 


Road 


4.4 


E 


Saddle 


4.5 


c 


Spur/Finger 



Table E.l Map Test Answer Key 



The GuUford-Zimmerman Aptitude Survey (Appendix E.6) assesses an 
individual’s spatial orientation ability. The results of this test are compared to a pool of 
national test scores to determine if a participant is above or below the national average 



144 



for spatial orientation. These results were used to determine which training phase a 
participant would undergo. 

The Practice Model Test (Appendix E.7) is administered to each virtual 
environment participant prior to moving onto the actual course model. It is used to 
ensure that a participant understands and is able to implement the interface functions. 
Each virtual environment participant was required to complete each task of the Practice 
Model Test. After completing the test, a participant is retested on any functions they 
failed to properly employ until he is able to do so. 

The Debriefing Questionnaires (Appendices E.8 and E.9) are administered prior 
to the final review of the participant’s route. Participants in the Real World and Map 
Only Group received the questionnaire in Appendix E.8. Virtual Environment 
participants receive the questionnaire in Appendix E.9 that has an additional page 
containing questions related to the virtual environment and its interface. The questions 
are designed to provide a qualitative analysis of the training materials and course. A five 
point scale (1-5) is used for the questionnaire. The final page of the questionnaire is 
designed to discover those terrain features an individual deems are most needed in a 
virtual environment from which they are obtaining navigational information. One item is 
deliberately left off the list of possible water features to see if participants are paying 
close attention or simply checking items on the list. This feature is streams/rivers. 

The raw scores from these tests and questionnaires are listed in Appendix O. 
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2. LAND NAVIGATION QUESTIONNAIRE 



Name: Age: Sex:, 

Branch of Service: Rank: 



1) Where did you first learn to navigate? 

a) Scouting, Boys/Girls Club 

b) Parents 

c) Friend 

d) ROTC/Academy 

e) Basic Training 

f) Officer Candidate School 

g) Officers Basic Course 

h) Other: 

2) How many years have you been Orienteering^’Navigating? 

a) less then a year 

b) one year or more 

c) two years or more 

d) five years or more 

e) ten years or more 

3) At what level would you classify your navigating abilities? 

a) Novice/Beginner 

b) Intermediate/Average 

c) Expert/Advanced 

4) How many Land navigation or Orienteering courses have you done in the last year? 



5) The land navigation course runs through varying degrees of vegetation and over 
rolling terrain. It will require you to negotiate a distance of no more than three miles 
in one hour. Do you have any physical disabilities that would prevent you from 
executing this task? Yes/No 



146 



3. SELF ABILITY EVALUATION 



Participant ID: 



The following bar line depicts the navigation ability evaluation of an average 
infantry officer with five years experience. The “X” indicates his ability level. 

X 



Knows how to Navigates with 

read a map no errors; 

Rarely looks 
at map 



Place an “X” on the line below were you feel your navigational abilities are at this 

time. 



I 

Knows how to 
read a map 



1 

Navigates with 
no errors; 
Rarely looks 
at map 
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4, SANTA BARBARA SENSE-OF-DIRECTION SCALE 



Participant ID: Date: SEX: F M AGE: 

This questionnaire consists of several statements about your spatial and 
navigational abilities, preferences, and experience. After each statement, you should 
circle a number to indicate your level of agreement with the statement. Circle “1” if you 
strongly agree that the statement apphes to you, ‘7” if you strongly disagree, or some 
number in between if your agreement is intermediate. Circle “4” if you neither agree nor 
disagree. 

1. I am very good at directions. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

2. I have a poor memory for were I left things, 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

3. I am very good at judging distances. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

4. My “sense of direction” is very good. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

5. I tend to think of my environment in terms of cardinal directions (N, S, E, W) 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

6. I very easily get lost in a new city. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

7. I enjoy reading maps. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

8. I have trouble understanding directions. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

(turn over and continue) 
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9. lam very good at reading maps. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

10. 1 don’t remember routes very weU while riding as a passenger in a car. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

1 1. 1 don’t enjoy giving directions. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

12. It’s not important to me to know where I am. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

13. 1 usually let someone else do the navigational planning for long trips, 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

14. 1 can usually remember a new route after I have traveled it only once, 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

15. 1 don’t have a very good “mental map” of my environment, 

strongly agree 1 2 3 4 5 6 7 strongly disagree 
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5. MAP READING TEST 



The foUowing is a list of terrain features commonly found on military and/or 
orienteering maps. Using the list of terrain features, identify the most predominate 
terrain feature within each circle and place your answer in the space provided. Each 
terrain feature from the list may be used more than once or not at all. 



A. Depression 

B. Draw 

C. Hill Top 

D. Lake/Pond 

E. Ridge Line 

F. Road/TraU 

G. Road/Trail Intersection 

H. Saddle 

I. Sour/Finger 

J. Stream/River 

K. Stream/River Intersection 

L. VaUey 




1. 2. 3. 4. 5. 
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A. Depression 

B. Draw 

C. Hill Top 

D. Lake/Pond 

E. Ridge Line 

F. Road/Trail 

G. Road/Trail Intersection 

H. Saddle 

I. Spur/Finger 

J. Stream/River 

K. Stream/River Intersection 

L. Valley 




1 . 2 . 3 . 4 . 5 . 
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A. Depression 

B. Draw 

C. HUlTop 

D. Lake/Pond 

E. Ridge Line 

F. Road/Trail 

G. Road/Trail Intersection 

H. Saddle 

I. Spur/Finger 

J. Stream/River 

K. Stream/River Intersection 

L. Valley 




1 , 2 . 3 . 4 . 5 . 
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Using the following map representations, choose the best representation for each 
picture displayed below. The map representations are a facsimile of the terrain shown in 
the photos. Some map representations may be used more than once or not at all. 
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6. GUILFORD-ZIMMERMAN APTITUDE SURVEY 




The GuiUord-Zimmerman Aptitude Survey 




Part 5/Spatial Orientation 



Ccpftyiii SopjSr Co- CA 

Ah tcspKi. /.'i;;* .•;) U refrodvasi .fl p 

53^7 iTJCtS i'rwisw (»3.'ftcr cwA^-u<r 
tv Coss:<rfi^ Ps>c^x>ic9isis rtm inc. 



Name 



Date Score — : Sex: M F 



INSTRUCTIONS 

This is a test of your ability to see changes in direction and posiUon. In each item you are to note how the 
position of the boat has changed in the second picture from the original position in the first picture. 



Here Is Sample Item 1. 

These bars represent the boat's 
prow. 



This is the correct answer. It shows 
that the p>row of the boat has drop- 
ped below the aiming point. 



(U^the prow had risen, instead of 
dropped, the correct answer would 
have been C, instead of D.) 



These are the live possible answers to the item. 




This is the prow (front end) of a 
nnotor boat in which you are riding. 

This Is the airrtng point. It is the 
exact spot you would see on land 
H you sighted right over the point 
of the prow. 



This is the same aiming point 
shown above. Note that the prow 
has dropped below it. 



To work caoh item: First, look at the lop picture and see where the motor boat is headed. Second, look at the 
bottom picture and fX)te the CHANGE in the boat's heading. Third, mark the answer that showsThe same change on 
the separate answer sheet. 



Try Sample Item 2. 

This also shows that the prow of 
the boat Is to the right of the aiming 
point. So. it is the correct answer. - 



(If the boat had lurned to the left, 
instead of to the right, the correct 
answer wouW have been A.) 




Consulting Psychologists Press, tnc.. 3803 E. Bayshore Road, Palo Alto, CA 94303 
98 97 96 9S94 87654 



0039 



Figure E.l Guilford-Zimmerman Aptitude Survey Cover Page 
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7. PRACTICE MODEL TEST 



a. Turn to a heading of 360 degrees and begin movement. 

b. Switch to a top down view 

c. Switch to a 15-meter view 

d. Change to run mode 

e. Change to walk mode 

f. Move to the road and take a right 

g. While following the road: 

i) Look-up 

ii) Look down 

iii) Look left 

iv) Look Right 

h. Head into town 

i. Stop 

j. What is your heading? 

k. Begin movement. 

l. Run 

m. Slow down and stop at the road sign 

n. Look to your right. What do you see? 

o. Using the quick view keys, see what is at CP6 

p. Using the hot keys, return to the start point 
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8. DEBRIEFING QUESTIONNAIRES 

a. Map and Real World Group Debriefing Questionnaire 



MAP 


Hard to Read 






Easy to Read 


Was the map easy to read? 


1 2 


3 


4 


5 N/A 


Was the map easy to understand? 


Hard to Understand 
1 2 


3 


4 


Easy to Understand 
5 N/A 


Were the trails & roads adequately 
shown on the map? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Were the man made structures 
adequately shown on the map? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Were the obstacles adequately 
shown on the map? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Was the vegetation adequately 
shown on the map? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Using the map, ho\\ difficult was it 
to plan your route? 


Easy 

1 2 


3 


4 


Very Difficult 
5 N/A 


Comments: 


COURSE 1 


Easy 






Very Challenging 


How difficult was the course? 


1 2 


3 


4 


5 N/A 


Were the control points well marked? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Were the control points located 
where you expected them? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Had routes been trampled down 
leading to the control points? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Did you have difficulties remembering 
your planned route? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Comments: 


MISC 


Definitely Not 






Definitely Yes 


Did you enjoy this experiment? 


1 2 


3 


4 


5 N/A 


Did you feel the training phase 
was long enough? 


Definitely Not 
I 2 


3 


4 


Definitely Yes 
5 N/A 


Did you feel the training phase 
was too short? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Do you feel the training familiarized 
you learn the environment? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Did you feel confident in navigating 
the terrain without a map or compass? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Comments: 
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1. Place an "X" next to the items you feel must be replicated in a model that prepares you to navigate an 
actual piece of terrain. 



buildings 


factory 


buildings 


houses 


buildings 


public buildings 


buildings 


shacks 


buildings 


other 



misc 


compass 


misc 


road signs 


misc 


rock piles 


misc 


sand bags 


misc 


street signs 


misc 


the sun 


misc 


people 


misc 


animals 


misc 


sound 


misc 


other 


misc 


other 


terrain 


clearings 


terrain 


depressions 


terrain 


hills 


terrain 


knolls 


terrain 


ridgelines 


terrain 


spurs/fingers 


terrain 


other 



roads 


dirt roads 


roads 


foot paths 


roads 


paved roads 


roads 


trails 


roads 


other 



obstacles 


electric lines 


obstacles 


pits/fox holes 


obstacles 


shallow ditches 


obstacles 


telephone poles 


obstacles 


towers 


obstacles 


trenches 


obstacles 


other 



vegetation 


bushes 


vegetation 


flowers 


vegetation 


grass/weeds 


vegetation 


trees 


vegetation 


other 



water 


lakes 


water 


marsh lands 


water 


ponds 


water 


puddles 


water 


swamps 


water 


other 



2. From the list of items in question # 1, choose and rank the six items you feel are the most impcjrtant 
for a computer model which will be used to prepare an individual to navigate an actual piece of terrain. 



1 

2 

s' 

4 

5 

6 
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b. Virtual Environment Group Debriefmg Questionnaire 



MAP 


Hard to Read 






Easy to Read 


Was the map easy to read? 


1 2 


3 


4 


5 N/A 


Was the map easy to undCTstand? 


Hard to Understand 
1 2 


3 


4 


Easy to Understand 
5 N/A 


Were the trails & roads adequately 
shown on the map? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Were the man made structures 
adequately shown on the map? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Were the obstacles adequately 
shown on the map? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Was the vegetation adequately 
shown on the map? 


Definitely Not 

1 2 


3 


4 


Definitely Yes 
5 N/A 


Using the map, how difficult was it 
to plan your route? 


Easy 

1 2 


3 


4 


Very Difficult 
5 N/A 


Comments: 


COURSE 1 


Easy 






Very Challenging 


How difficult was the course? 


1 2 


3 


4 


5 N/A 


Were the control points well marked? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Were the control points located 
where you expected them? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Had routes been trampled down 
leading to the control points? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Did you have difficulties remembering 
your planned route? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Comments: 




MISC 1 


Definitely Not 






Definitely Yes 


Did you enjoy this experiment? 


1 2 


3 


4 


5 N/A 


Did you feel the training phase 
was long enough? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Did you feel the training phase 
was too short? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Do you feel the training familiarized 
you learn the environment? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Did you feel confident in navigating 
the terrain without a map or compass? 


Definitely Not 
1 2 


3 


4 


Definitely Yes 
5 N/A 


Comments: 
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MODEL 


Definitely Not 






Definitely Yes 


Was the model clear and viewable? 


1 2 


3 


4 


5 N/A 




Definitely Not 






Definitely Yes 


Did the model coincide with the map? 


1 2 


3 


4 


5 N/A 


Were the trails & roads adequately 


Definitely Not 






Definitely Yes 


represented in the model? 


1 2 


3 


4 


5 N/A 


Were the man made structures 


Definitely Not 






Definitely Yes 


adequately represented in the model? 


1 2 


3 


4 


5 N/A 


Were the obstacles adequately 


Definitely Not 






Definitely Yes 


represented in the model? 


1 2 


3 


4 


5 N/A 


Was the vegetation adequately 


Definitely Not 






Definitely Yes 


represented in the model? 


1 2 


3 


4 


5 N/A 


Were changes in elevation adequately 


Definitely Not 






Definitely Yes 


represented in the model? 


1 2 


3 


4 


5 N/A 


Did the model help you identify the 


Definitely Not 






Definitely Yes 


control points within the last 50m? 


1 2 


3 


4 


5 N/A 


Did the model help you identify the 


Definitely Not 






Definitely Yes 


general area of the control points? 


1 2 


3 


4 


5 N/A 


Using the model, how difficult was it 


Easy 






Very Difficult 


to plan your route? 


1 2 


3 


4 


5 N/A 


Do you feel the model gave you an 


Definitely Not 






Definitely Yes 


advantage you normally wouldn’t 


1 2 


3 


4 


5 N/A 


have had? 










Would you use this tool if it were 


Definitely Not 






Definitely Yes 


available for mission planning? 


1 2 


3 


4 


5 N/A 


Would you use this tool if it were 


Definitely Not 






Definitely Yes ' 


available for mission rehearsal? 


1 2 


3 


4 


5 N/A 


Would you use this tool if it were 


Definitely Not 






Definitely Yes 


available for navigation training? 


1 2 


3 


4 


5 N/A 


Comments: 


MODEL INTERFACE 


Confusing 






User Friendly 


Were you able to easily move through 


1 2 


3 


4 


5 N/A 


the model? 












Confusing 






User Friendly 


Was the joystick easy to use? 


1 2 


3 


4 


5 N/A 




Confusing 






User Friendly 


Was the acceleration lever easy to use? 


1 2 


3 


4 


5 N/A 




Confusing 






User Friendly 


Were the toggle buttons easy to use? 


1 2 


3 


4 


5 N/A 




Confusing 






User Friendly 


Your overall felling about the interface? 


1 2 


3 


4 


5 N/A 


Was the 15-minute train-up on the 


Definitely Not 






Definitely Yes 


initial model useful? 


1 2 


3 


4 


5 N/A 


Was the 15-minute train -up on the 


Definitely Not 






Definitely Yes 


initial model enough time to become 


1 2 


3 


4 


5 N/A 


familiar with the interface? 










Did the use of three screens cause 


Definitely Not 






Definitely Yes 


any confusion when maneuvering? 


1 2 


3 


4 


5 N/A 


Comments: 
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1 . Place an "X" next to the items you feel must be replicated in a model that prepares you to navigate an 
actual piece of terrain. 



buildings 


factory 


buildings 


houses 


buildings 


public buildings 


buildings 


shacks 


buildings 


other 



mi sc 


compass 


misc 


road signs 


misc 


rock piles 


misc 


sand bags 


misc 


street signs 


misc 


the sun 


misc 


people 


misc 


animals 


misc 


sound 


misc 


other 


misc 


other 


terrain 


clearings 


terrain 


depressions 


terrain 


hills 


terrain 


knolls 


terrain 


ridgelines 


terrain 


spurs/fingers 


terrain 


other 



roads 


dirt roads 


roads 


foot paths 


roads 


paved roads 


roads 


trails 


roads 


other 



obstacles 


electric lines 


obstacles 


pits/fox holes 


obstacles 


shallow ditches 


obstacles 


telephone poles 


obstacles 


towers 


obstacles 


trenches 


obstacles 


other 



vegetation 


bushes 


vegetation 


flowers 


vegetation 


grass/weeds 


vegetation 


trees 


vegetation 


other 



water 


lakes 


water 


marsh lands 


water 


ponds 


water 


puddles 


water 


swamps 


water 


other 



2. From the list of items in question # 1, choose and rank the six items you feel are the most important 
for a computer model which will be used to prepare an individual to navigate an actual piece of terrain. 



1 

i 

3 

4' 

5' 

6 ' 
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APPENDIX F. COURSE 



1. GENERAL 

This appendix consists of six items: 1:50,000 map excerpt of the course area, 
1:24,000 map excerpt of the course area, an aerial photo of the course, an aerial photo 
with an example participant debriefing route, 1:5,000 course orienteering map, and an 
explanation of the map legend [BANK 97]. The 1:50,000 and 1:24,000 maps are the 
standard scales used by most US ground forces for military operations. Comparison with 
the course orienteering map show the magnitude of the additional information which can 
be gleaned from the orienteering map as compared to even the high resolution 1:24,000 
military operations map. The aerial photo is the same one utilized by MAJ Banker to 
produce the original course map and was also used to display the participant’s route 
during the debriefing phase of the experiment. The course map was modified from the 
original one developed by MAJ William Banker after field verification by CPT Simon 
Goerger. The map legend explanation is taken directly from Appendix D of MAJ 
Banker’s 1997 Masters Thesis. 

2. 1:50,000 MAP EXCERPT OF COURSE AREA 

The center of Figure F. 1 is the course area. The boundary roads and two north 
south trails are the only liner features that can be depicted on this map for the area. The 
high ground in the southwest comer of the course and the low ground on the east edge of 
the course are the only discemable terrain features. The entire course is depicted as being 
wooded. A 1:50,000 map of the Fort Ord training area was used to verify that a 
participant had not been in the target area in the past. Its lack of detail and the general 
overview it provided of the training area made it possible to identify locations where 
participants may have explored the old Fort Ord training area without furnishing 
participants additional information about the orienteering course. 




Figure F.l 1:50,000 Map Excerpt of Course Area (Actual Size) 
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3. 1:24,000 MAP EXCERPT OF COURSE AREA 

The center of the Figure F.2 is the course area. The boundary roads and two north 
south trails are the only liner features that can be depicted on this map. The high grounds 
in the southwest comer and east of the course are discernable terrain features as well as 
the low ground on the east edge of the course and the northwest comer of the course. 
The entire course is depicted as being wooded. 




Figure F.2 1:24,000 Map Excerpt of Course Area (Actual Size) 
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4. AERIAL PHOTO 




Figure F.3. Aerial Photo 
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5. AERIAL PHOTO WITH PARTICIPANT ROUTE 



Figure F.4. Aerial Photo With Subject Route 
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6. COURSE MAP 




Figure F.3. Course Map 
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7. COURSE MAP LEGEND EXPLANATION 



All maps are generalizations. They use symbols to portray actual features on the 
earth’s surface. Not all features are represented with the same precision. Discrete non- 
vegetation items are plotted on the map in the exact location they are in the actual 
environment, whereas vegetation boundaries (unless indicated with a distinctive dotted 
line) are not meant to represent a clean break from one type of vegetation to another. 
Rather, this line separating one vegetation area from another is a generalization of where 
one type more or less ends and another more or less begins. The line separating the two 
can best be thought of as a blurry line where the two types of vegetation intermingle. The 
below guide will help to determine the specific limitations of each symbol on the 
orienteering map. 

Building - Buildings in the area are of several types: 

a. Latrines - most common building, tan in color, approx, size 3x8 meters 

b. Shelters - second most common building, green wood, roofed, no walls, 

approx, size 3x8 meters 

c. Admin. - field office and shack, black with gold trim, 8x8 meters and 2x2 

meters respectively 

Open Sandy Ground - a significant patch of sand that will slow running 

Open ground - dirt, hard pack, free of grass and other vegetation. 

Undergrowth walk - immature chaparral or oak, dense stands of bushes, 
incomplete overlap of two distinct areas of fight which allow restricted passage along that 
overlap, other plants that prevent running. 
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Fight - mature chaparral or immature oak in such density that passage through is 
very difficult, running impossible 

Forest walk - oak forest with patchy undergrowth, low lying tree limbs or tree 
density that prevents running fi-om being sustained 

Forest slow run - oak forest fairly fi'ee of undergrowth, but with low lying limbs 
or tree density that makes sustained running difficult. 

Rough open ground - grass covered ground, possibly with scattered (avoidable) 
undergrowth. Note that there are a few locations that have what appears to be old jeep 
trails but are portrayed as rough open ground. Sometimes the distinction between one or 
the other blurs. If in doubt refer to other more distinctive features (contour lines, etc.) to 
determine your location. 

Shallow depression - most likely an old decaying foxhole position or other man 
made excavation where the banks have eroded to create a bowl-like depression of 1 to 3 
feet below surrounding ground. 

Misc. object - a manmade feature, rubble, derelict military equipment, or other 
item whose exact description is only provided if it is the location of a control 

Pit - an old foxhole or likely other man made pit that has steep vertical walls and 
may be reinforced with wood, depth firom 2 to 5 feet. Note that there will be many pits in 
the area that are not depicted on the map. The pits that are depicted are accurate. 

Telephone poles - wood poles (if bearing wire it will be noted on map) approx. 
25 to 30 feet in height 

Concrete pad - old concrete tent pad extending firom 2 to 5 inches above ground 

level 
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Tree - a tree or large bush (could be two or more trees growing close together - 
forming an unbroken single canopy — if the trees are smaU) 

Rootstock - a dead or overturned tree 

Troop training device - a bunker or other man made item built for training 
soldiers 

Vegetation boundary - the edge of a vegetation type 

Gully or Ditch - ranging from a shallow 1-foot deep gully to 5-foot deep military 

trench 

Jeep Trail - a road more suitable for 4 x 4 vehicles due to width restriction and/or 
ruts. May be distinctive and worn or in some places overgrown with grass but still 
containing ruts. 

Paved Road - a surfaced all weather road 

Road - a sandy or dirt road wide and level enough for 2 wheel drive vehicles 

Indistinct Path - a path that is in the process of being overgrown with only 
intermittent marks on the ground that indicate that it was once a well traveled path 

Narrow Ride - a linear break in the forest that may have once been a jeep trail 
but now is overgrown with grass and lacks telltale wheel ruts 

Path - a foot or bike path. 
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APPENDIX G. PARTICIPANT TASK LIST^ 

Thank-you for participating in this study. You will do an Orienteering course 
today. However, there are some important differences to note: 

1. You will be wearing a light pack with DGPS and Newton MSG Pad 130. Its purpose 
is to log your route and act as a data capture device for other actions you may perform. 

2. Before you run the course you will carefully plan your route through the entire course 
(see Important Information on Marking Your Map) 

3. Use this training time to commit the route and course to memory. You are expected to 
do the following on the actual course run: 

a. Navigate without aid of map and compass, utilizing only your memory 

b. Attempt to find all the controls utilizing your planned route 

Summary of objectives 
All Objectives are equally important!! 

1 . Choose the most efficient route based on your abilities 

2. Minimize the number of map checks you request fi'om the administrator 

3. Minimize the number of compass checks you request from the administrator 

4. Minimize the number of map with compass checks you request firom the 
administrator 

5. Stay on your planned route 

6. Find all the controls in order (you have 60 minutes to conduct this task) 

• If you need to make a map check then say so and the administrator will give you the 
map for 30 seconds. Additional time can be requested in 30 second increments at the 
additional cost of a map check each. 

• If you need to make a compass check then say so and the administrator will give you 
the compass for 30 seconds. Additional time can be requested in 30 second 
increments at the additional cost of a compass check each. 

• If you need both map and compass then say so and the administrator will give you 
both for 60 seconds. Additional time can be requested in increments of 60 seconds. 

• If you want to change your route announce to the administrator that you are changing 
your route plan. At that point the administrator will hand you the map, compass, and 
blue pen. From the time that he gives you the materials you will have 30 seconds to 
plot the new route. If you need more time then tell him you need more time and you 
will get another 30 seconds. Request additional time as needed but remember that 
one of your objectives is to make as few map checks as necessary. Every 30 seconds 
that you are looking at the map beyond the original 30 seconds for the route 
change counts as a map check. 



* This document is adapted and modified firom MAT Banker’s Masters Thesis [BANK 98] 
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APPENDIX H. MAP MARKING INSTRUCTIONS^ 

Pay close attention to how you mark your route, be as precise as the map and pen 
allow. Before your actual run you are expected to preview your map within your group’s 
prescribed context. Mark your planned route using the RED pen. You may correct 
any mistakes you make while planning with the white eraser. Once the planning period is 
up or you elect to finish you will not be allowed to erase any of the red route marks you 
have made. SO BE PRECISE in marking your map, detail does matter. Later during 
the actual course run anytime that you are going to deviate from your planned route you 
must stop: 

1 . Announce to the administrator that you are changing your route plan. At that point the 
administrator will hand you the map. From the time that he gives you the map you wUl 
have 30 seconds to plot the new route. If you need more time than tell him you need 
more time and you will get another 30 seconds. Request additional time as needed but 
remember that one of your objectives is to make as few map checks as necessary. Every 
30 seconds that you are looking at the map beyond the original 30 seconds for the 
route change counts as a map check. 

2. Take the blue pen and draw in your new route with the same attention to detail that you 
applied or the original route planning in red. 

3. Leave your original route on the map. The eraser is provided so that you may make 
corrections to a route as you draw it. Once you finish drawing and begin navigating you 
are not allowed to erase routes, or corrections to planned routes (blue penned routes). 

4. You may make as many corrections to your route(s) as necessary while navigating the 
course. 



Importance of detail in map marking and navigation 

You are allowed to deviate from your planned route within the following 
tolerances while still being considered on that route: 

Jeep Trails, Paved Roads, Unpaved Roads, Indistinct Paths, Narrow Rides and 
Paths — If your marked route is on any of these features you are allowed 5 meters either 
side of the feature and you are still considered as being "on your route". 

All other features — On all other types of non road/trail terrain you may travel 15 
meters to either side of your marked route and you are still considered as being "on your 
route" 



*This document is adapted and modified from MAJ Banker’s Masters Thesis [BANK 98] 
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APPENDIX I. DIGITAL PHOTOS 



1. GENERAL 

Subjects are provided with a series of digital images of the control points. Map 
and Real World subjects receive the photos displayed in Appendbc 1.2 while Virtual 
Environment subjects receive the Appendix 1.3 photo sets. The photos are furnished in 
color. The photos help to outfit the subject with a stronger grasp of the defining 
landmarks they are searching for. Under conditions that would allow the production of 
such a detailed map of the area, it is feasible to expect that reconnaissance photos would 
be available of these locations. 

The VE participants are also presented with screen capture images of the Control 
Points from the same general direction and distance as the actual photos were taken. This 
provides the VE participants with additional information to assist them in resolving the 
differences between the VE and the real world. 
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2. MAP AND REAL WORLD GROUP PHOTOS 





Control Point 4 



Control Point 1 




Control Point 2 




Control Point 3 



Control Point 5 




Control Point 6 
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Control Point 7 




Control Point 8 




Control Point 9 
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3. VIRTUAL ENVIRONMENT GROUP PHOTOS 



Real World 




Control Point 1 




Control Point 2 




Control Point 3 



Model 




Control Point 1 




Control Point 2 




Control Point 3 
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Real World 



Model 





Control Point 4 



Control Point 4 




Control Point 5 




Control Point 6 




Control Point 5 




Control Point 6 



177 







Real World 




Control Point 7 




Control Point 8 



Model 




Control Point 7 




Control Point 8 




Control Point 9 



Control Point 9 
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APPENDIX J. COURSE EQUIPMENT CHECKLIST 



Binder Containing: 

Subject’s map & designated route 

Think Out Loud Instructions 

Data Collection Sheet 

Researcher’s Script 

Data Recording: 

blue alcohol pen to record route deviations 

red pen to record data 

digital camera 

helmet & 8mm camera 

rucksack frame w/GPS system 

stop watch/timer 



Misc: 



extra battery (8 mm camera) 

extra cassette (8mm camera) 

extra Color Wheels for Tasks 3.1. & 5.1 

extra arrows (color wheels) 

extra clue sheet (incase subject looses his/hers) 

blindfold (for movement to course) 

cellular phone (optional) 

compass 

first aid kit 

Tecnu (for poison oak) 

water 

Prepositioned: 

Color Wheel Platform for Tasks 3.1. & 5.1 

Control flags 
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APPENDIX K. TfflNK OUT LOUD INSTRUCTIONS^ 

Your thoughts are important to this research. As you navigate the course you 
should be “thinking out loud”. 

As you move through the environment and experience it directly express what 
you are thinking. The mental preconception you had of this environment before you 
stepped into it will now be evaluated by you as you experience the course directly. As 
this image is confronted with direct experience your expectations and plan may be 
confirmed, modified, or refuted. Be sure to talk out loud these thoughts. 

The process of talking out loud and paying close attention to your route will slow 
you down. This is expected and why you are given an hour to finish the course. 



PLEASE SPEAK LOUDLY SO THAT YOUR VOICE WILL BE PICKED UP BY 

THE MICROPHONE 



^ This document is adapted and modified from MAJ Banker’s Masters Thesis [BANK 98] 



APPENDIX L. ROUTE CLASSIFICATIONS 

1. GENERAL 

This appendix consists of five items: route analysis, an explanation of route 
classifications for each leg of the course [BANK 97], route classifications based on a 
LISP Program, participant route classifications based on MAJ Banker’s route 
classification and on the LISP generated routes, and optimal route plan for movement 
from Control Point 9 to Control Point 4. The explanation of route classifications for each 
leg of the course is taken directly from Appendix F of MAJ Banker’s 1997 Masters 
Thesis. 

Route classifications were utilized to categorize the difficulty of an individual’s 
planned routes for comparison to their navigational ability. Routes were classified using 
MAJ Banker’s route classification listing (Appendix L.3) and again utilizing the results 
of a LISP route planning program (Appendix L.4). 

2. ROUTE ANALYSIS 

Participant routes were analyzed for difficulty level and performance. 
Participants’ Leg Error Scores were correlated with their Leg Difficulty Rating and 
ability level. A simple analysis of ability level to planned route difficulty shows that 
participants with higher GZ Scores and a high-perceived level of navigation ability 
planned simpler routes (Figure L.l). 



Cell Line Chart 
Grouping Variable(s): Bar £Xal 
Split By: GZ Ability Group 
Error Bars: ± 1 Standard BrrOT(s) 

Row exclusion: Data.snxsvd 

!_• I I I . I • I I I • L 



Figure L.l. Group vs LISP Planned Route Difficulty by Guilford-Zimmerman Scores 



Intermediate 



Beginner 



Expert 




.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 

Cell Mean for USP Average Planned Route Difficulty Level 
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This suggests that individuals with higher spatial ability have the ability to 
recognize desirable landmarks on the course and plan more conservative routes to locate 
those landmarks in route to their objective. Participants who ran short of time during the 
study phase due to becoming disoriented in the training environment or failed to 
maximize the tools they were afforded, hastily planned their final legs which usually 
resulting in an azimuth and distance approach to the problem. A straight distance and 
azimuth usually forced participants to negotiate thickly vegetated terrain, in which they 
became entangled and veered off their intended course, resulting in an increased number 
of errors. 

Figure L. 1 suggests that routes generated by a LISP program may provide us with 
the ability to predict routes would be best suited for a group based on the team’s spatial 
abilities and navigational experiance. If participants plan routes, which are more difficult 
than thek ability level (Appendix L, Section 4), the chances they will fail to successfully 
execute the planned routes increases while intermediate navigators who plan routes closer 
to the beginner level than the intermediate level also find fewer controls (Figure L.2). 
Advanced navigators plan routes just below the intermediate level and perform very well 
as they plan and operate within their abilities. 
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The same results are present when conparing performance based on ability levels 
and ISOM Average Route Difficulty Levels (Figure L.3). 



Scattergram 

Split By: Bar Eval 

Row exclusion: Data_snLsvd 



• Beginner 
O Expert 
^ Intermediate 



1 1.2 1.4 1.6 1.8 2 2.2 24 2.6 2.8 

ISOM Average Planned Route Difficulty Level 

Controls Found = 8.041 - .953 * ISOM Average Planned Route Difficulty Level; R'^2= .038 (Intermediate) 




Figure L.3. Performance Based on Ability and ISOM Average Route Difficulty 
Conversely, if a program can generate a preplanned route through an environment 
with respect to mission requirements and individual navigational abilities, we can reduce 
the mission planning time by military personnel and concentrate on mission rehearsal. Of 
course, such routes must be reviewed, modified, and verified by the personnel conducting 
the operation to ensure they understand and feel comfortable with the route. 

Further research needs to be conducted to determine if we can accurately predict 
an individual’s navigational performance based on their abilities and the difficulty level 
of planned routes. This research depends on our ability to evaluate an individual’s 
navigational ability and produce a program that can plan viable routes based on mission 
requirements and terrain characteristics. The LISP program in Appendix L.4 is a rough 
draft attempt which takes into account many of the aspects which must be considered if 
such a route planning tool is to be developed. 



3. BANKER’S ROUTE CLASSIFICATIONS 



What follows is MAJ Banker’s classification of some of the most probable routes 
to a given control and is based on the International Specification for Orienteering Maps 
[INTE 90]. They do not represent the only ways of getting to a control but the most 
likely routes chosen by participants based upon MAJ Banker’s orienteering experience 
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and knowledge of the terrain. The classifications are used as a basis for comparison with 
the routes selected by the LISP Route Selection Program. All controls possess at least 
one beginner’s route. The proportion of handrails to catching features delineates 
intermediate and advanced routes. If there are more handrails as compared to catching 
features then the route is intermediate. The opposite is true if there are more catching 
features to handrails. Utilizing MAJ Banker’s method to classify routes taken by 
participants, if an exact match for a participant’s route could not be found from the below 
list, the route was examined within the context of its use of handrails (including what 
type) and catching features and assigned a route designation. This designation correlates 
V.' -h the same level of difficulty for the >utes on that control (beginner, intermediate, 
advanced) [BANK 97]. 
a. Control 1. 

1. Beginner 

a) Gigling Road west to jeep trail 

b) Jeep Trail south by east by south to building 

c) Control on NW comer of building 

2. Beginner 

a) Watkin's Gate Cutoff to indistinct path. 

b) Indistinct path southwest up hill to jeep trail 

c) Jeep Trail west to building 

d) Control on NW comer of building 

3. Intermediate 

a) West through plotted individual trees (catching features) 

b) Handrail rough open ground south to junction indistinct 
path and jeep trail 

c) Jeep Trail west to building (catching feature) 

d) Control on NW comer of building 

4. Advanced 

a) West through plotted individual trees 

b) Follow mnnable forest southwest 

c) Try to hit small rough open gap by keeping walkable forest 
to left shoulder 
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d) Use forest fight to west as catching feature if needed 

e) Control on NW comer of building 

f) Use jeep traU for catching feature if control is missed 

5. Advanced 

a) Go straight at control from start 

b. Control 2. 

1 . Beginner 

a) Jeep trail northwest to budding 

b) Follow open ground to west and look for rough open 
clearing going northwest (handrail) 

c) Follow rough open clearing northwest looking for pit 

d) Control in pit 

2. Intermediate 

a) Jeep trail northwest to building 

b) Go straight at control (WSW) from building 

3. Advanced 

a) Set out on straight line directly for control 

b) Hit open ground and look for building on the right and 

rough open break on the left. (Catching feature) 

c) Follow rough open clearing northwest looking for pit 

d) Control in pit 

c. Control 3. 

1. Beginner 

a) Head northwest and get out onto Gigling Road 

b) Take Gigling Road west to jeep trail junction with 
telephone pole 

c) Take jeep trail southeast to convergence of two jeep trails 

d) Head southwest into tree grove looking for control 

(1) Use building as catching feature 

(2) Use open ground to west as backup catching feature 

e) Control hanging from tree limb 

2. Advanced 
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a) Head straight at control; use jeep trail prior to control as 
catching feature 

b) Head southwest into tree grove looking for control 

(1) Use building as catching feature 

(2) Use open ground to west as backup catching feature 

c) Control hanging from tree limb 
d. Control 4. 

1. Beginner 

a) Head southwesterly and try to get on jeep trail headed in 
same direction 

b) Take jeep trail to junction 

c) Take jeep trail southeast to junction 

d) Take southerly fork to next junction 

e) Take fork to northwest 

f) Once beyond patches of fight leave trail and start looking 
for control 

g) Control is in pit 

2. Beginner 

a) Turn around and go back to jeep trail to the east 

b) Take jeep trail southwest to junction 

c) Take fork to the south to another junction 

d) Take fork to the west to next junction 

e) Take southerly fork to next junction 

f) Take fork to northwest 

g) Once beyond patches of fight leave trail and start looking 
for control 

h) Control is in pit 

3. Intermediate 

a) Go south towards road junction 

b) Get on road and take to junction 

c) Take road west to other road junction 

d) Handrail around fight to west coming down through small 

patch of fight into control 

4. Advanced 
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a) Head straight at control expect to hit jeep trail that runs 
NW to SE (catching feature) 

b) Hit trail and then thread way through scattered fight 

c) Emerge into center of depression and rough open ground, 
(catching feature) look for pit 

d) Control is in pit 

e. Control 5. 

1. Beginner 

a) Move back out onto jeep trail 

b) Take trail west to trail junction 

c) Take trail WNW up to misc object 

d) From misc. object go straight at control 

2. Intermediate 

a) Move directly at control 

b) Use Gigling Road as catching feature if miss on control 

c) Control is in center of clearing 

3. Advanced 

a) Move directly at control 

b) Use southwesterly linear clearing as catching feature 

c) Follow clearing NW right into control 

d) Use Runnable forest along Gigling as catching feature in 
case of miss 

f. Control 6. 

1 . Beginner 

a) Move out onto Gigling Road and take it westerly to 
junction with dirt road 

b) Move down dirt road (south) to junction with jeep trail 

c) Take jeep trail to east look for concrete rubble 

d) Move southeast through runnable forest 

e) Look for control on concrete pad 

2. Beginner 

a) Move straight at control and hit jeep trail 

b) Go southwest on Jeep trail to junction with another jeep 
trail 
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c) Take jeep trail westerly and look for concrete rubble 

d) Move southeast through runnable forest 

e) Look for control on concrete pad 

I 

3. Intermediate 

a) Move south to junction of two jeep trails (catching feature) 

b) Handrail jeep trail southeasterly to clearing (catching 
feature) 

c) Handrail clearing to the west 

d) Hit fight going west (catching feature) and move south 

e) Handrail fight (keeping it on right shoulder) into control 

f) Look for control on concrete pad 

4. Advanced 

a) Move straight at concrete rubble (aiming off technique) use 
jeep trail as catching feature and handrail 

b) Move southeast through runnable forest 

c) Look for control on concrete pad 

g. Control 7. 

1. Beginner 

a) Move back out onto east west jeep trail 

b) Go west to junction of jeep trail and dirt road 

c) Take dirt road south to junction with four jeep trails 

d) Take jeep trail east by northeast 

e) Look for second linear break in vegetation (indistinct path) 

f) Take indistinct path (handrail) to ditch 

g) Follow ditch to its end 

h) Control at east end of ditch 

2. Intermediate 

a) Move through rough open ground easterly to jeep trail 
(catching feature) 

b) Follow jeep trail (handrail) to junction with other jeep trail 
by building 

c) Locate telephone poles and follow wire (handrail) south 
easterly 

d) Hit fight and turn west and follow fight boundary into ditch 
(handrail) 

e) Control at east end of ditch 
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3. 



Advanced 



a) Move through rough open ground easterly to jeep trail 
(catching feature) 

b) Take jeep trail to curve where it turns east (hand rail) 

c) Leave jeep trail and head straight for control use east west 
jeep trail as checkpoint (catching feature) 

d) Aim off to east side of ditch and go southeast (telephone 
wires to east as catching feature to prevent drifting too far 
east) 

e) Use fight as catching feature 

f) Hit fight and turn west and follow fight boundary into ditch 

g) Control at east end of ditch 

4. Advanced 

a) Move straight at control 

b) Use jeep trail junction as attack point 

c) From attack point take offset route to west part of ditch 

d) Follow ditch to east and find control at end of ditch 

h. Control 8. 

1. Beginner 

a) Handrail fight to the east till hitting the jeep trail 

b) FoUow jeep trail northerly through intersection to sharp 
curve to the east 

c) Once at sharp curve to east turn off trail to west and look 
for control in clearing 

d) Control located in clearing 

2. Intermediate 

a) Handrail fight to telephone poles 

b) Take telephone poles NW back to jeep trail junction 

c) Follow jeep trails east to next junction 

d) Take jeep trail north 

e) Leave jeep trail and move directly at control 

3. Advanced 

a) Move directly at control (avoiding forest walk) use jeep 
trail junction as catching feature 
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b) From jeep trail junction aim off to east of control at sharp 
curve to east of jeep trail keeping eyes open for control in 
clearings 

c) Use same trail as Beginner route as catching feature (for 
drift) 

i. Control 9 (Finish) 

1. Beginner 

a) Move back out to jeep trail just to east of control 8 

b) Take trail south to four way junction with other trails 

(handrail) 

c) Take southeasterly running trail to trail fork 

d) Take northeasterly running fork to five way junction 
(handrail) 

e) Take northwesterly running trail keeping eyes open for 
small break in fight to the east (catching feature) 

f) Take indistinct path into clearing and hook to north 

g) Control on east edge of clearing 

2. Intermediate 

a) Move back out to jeep trail just to east of control 8 

b) Move off trail using rough open to move closer to control 

c) Take rough open out onto jeep trail which runs NE to SW 

d) Take trail to junction with North South jeep trail 

e) follow jeep trail looking for indistinct path 

f) Take indistinct path into clearing and hook to north 

g) Control on east edge of clearing 

3. Advanced 

a) Move straight at control on east by northeast azimuth 

b) Use trail as catching feature 

c) Fight to north and south of route used as catching features 

d) Lx)cate opening in fight 

e) Take indistinct path into clearing and hook to north 

f) Control on east edge of clearing 

4. LISP PROGRAM ROUTE CLASSIFICATION 

This program plans a route through a specified piece of terrain based on 
identifiable decision points and terrain characteristics. The information is manipulated by 
a branch and bound search, pruning heuristics, and terrain classification. The algorithms 
are coded in ANSI LISP. Because of the memory requirements of the search’s stacks and 
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the speed of the processors running the program, each leg was limited to passing through 
a maximum of eight decision points. This limitation forced the rejection of possible 
routes. 

Decision points are identified as any piece of terrain that would logically require 
an individual to make a decision on which direction to move. Although in a natural 
environment, a person on foot can move in almost any direction at any time, it was 
assumed that individuals would not intentionally change direction of movement unless 
they knew where they were and where they wanted to go. For this course, 99 decision 
points were identified. Decision points were associated with neighboring decision points 
based on proximity (Figure L.4). This meant that to traverse the course, the program had 
to link together neighboring decision points into a chain of successive segments to 
complete each leg of the course. 




Figure L.4. Decision Points and Neighbors 
The terrain between neighboring decision points is known as a segment. Each 
segment has a different point value based on a list of characteristics. This program 
utilized four factors (distance, mobility, observation, and difficulty of locating the next 
decision point) to determine segment values. Two additional factors which were not 
incorporated but which would have made the program more accurate are change in 
elevation and terrain revisited. 
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Distance between points is not assigned any additional weight in the program’s 
algorithm. Mobility is based on the difficulty of traversing the terrain. Mobility factors 
came directly from the terrain classification used on the orienteering map. Observation is 
also evaluated based of the orienteering map’s terrain classifications. Observation deals 
with the ability to see through or over the terrain’s vegetation. The final factor addressed 
is the issue of identifying when an individual has reached the decision point. Some 
decision points are easier to locate than others are. For example, it is more difficult to 
locate a control point placed in a pit as compared to a black shed in the middle of a 
clearing. The weights assigned for each of these factors was dependent on the ability 
level of the navigator (Tables L.l, L.2, and L.3). 



Ability Group 


Fight 


Walk 


Sand 


Run 


Open 


Road 


Beginner 


4.0 


2.5 


1.75 


1.5 


1.25 


1.0 


Intermediate 


3.0 


2.5 


2.0 


1.5 


1.25 


1.0 


Advanced 


2.0 


1.1 


1.075 


1.05 


1.0 


1.0 



Table L.l. LISP Program Mobility Weights 



Ability Group 


Forest 


Undergrowth 


Open 


Beginner 


4.0 


2.75 


1.0 


Intermediate 


3.0 


2.0 


1.0 


Advanced 


1.2 


1.1 


1.0 



Table L.2. LISP Program Observation Weights 



Ability Group 


Hard 


Moderate 


Easy 


Beginner 


5.0 


2.0 


1.0 


Intermediate 


3.0 


1.5 


1.0 


Advanced 


1.5 


1.2 


1.0 



Table L.3. LISP Program Identification Weights 

Weights were based on the impact of each element to the successful completion 
of a segment for each type of individual. The most difficult condition receives the 
highest weight. The lowest weight, easiest aspect, which could be assigned for any 
element is 1.0. For all individuals traversing the terrain on a road with open visibility to 
an easily identifiable decision point was weighted the same, 1.0. Conditions are rank 
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ordered from hardest to easiest and then assigned weights based on their position in the 
table. 

Mobility through the terrain was closely coupled for each group. This is based on 
observations that indicated little difference between abihty groups in the level of 
apprehensiveness of individuals who were faced with conducting cross-country 
movement. The most difficult terrain to cross. Fight, was weighted as a 4.0 for a 
beginner, 2.0 for an intermediate and 1.0 for and advanced navigator. These weights 
were based on the difficulty for beginners to maintain their course while traveling 
through fight, since they need constant verification that they are going in the correct 
direction. Intermediate navigators have fewer problems maintaining their course through 
difficult terrain but still require some assurance they are on the right course. Advanced 
navigators are more confident in their abilities, need less reassurance they are on the 
correct course and often plan their route to use catching features to confine their 
movement and halt their forward progress in the proximity of the next identifiable 
decision point. 

Visibility plays much less of a factor for advanced navigators than for 
intermediates or beginners. Intermediate and beginning navigators needs reassurance that 
they are on the right course. This is gained through many cues in the environment most 
of which are visual. Advanced navigators can gain reassurance through many senses 
such as the sound of a creek to the north or the warmth of the sun on the left side of their 
face. Because of this, navigators pay less attention to visual cues enroute to their 
objective as they confirm their position through the use of many input factors. This 
results in reducing the weight of the most cluttered environments to 1.2 for advanced 
navigators while intermediate navigators remain at 3.0 and beginners remain at 4.0. 

Identification of the decision point plays the most crucial part of the segment’s 
value for beginners. If beginning navigators cannot identify when they have reached the 
correct decision point, they often become confused or disoriented. This results in their 
becoming lost and losing confidence in their abilities to determine their location and 
continue their movement in a positive direction. Beginners also have greater difficulty 
choosing and identifying appropriate decision points since they continuously question 
their ability. This results in a greater chance of them misidentifying the correct decision 
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point if it is not an obvious one. For these reasons, the weight for hard to identify 
decision points for beginners is set at 5.0. Intermediate navigators have fewer problems 
choosing and identifying appropriate decision points. Since they do not question 
themselves as often as beginning navigators, they are less likely to incorrectly identify a 
decision point. The weight of hard to identify decision points for intermediate navigators 
is set at 3.0. Advanced navigators filter out much of the “noise” of an environment and 
often choose decision points they can readily identify, ignoring intermediate decision 
point enroute. For this reason they have less difficulty identifying the correct decision 
point. The weight for hard to identify decision points for advanced navigators is set at 
1.5. 

The weights for mobility, visibility, and identification are multiplied with the 
segment’s length. Each segment’s value is based on the resulting product. The route 
with the lowest value for its summed segments is chosen as the best route for that ability 
group. 

The program is designed to locate three optimal paths through the course. One 
Beginner (Figure L.5), one Intermediate (Figure L.6), and one Advanced Course (Figure 
L.7) are calculated and displayed on maps for comparison with participant maps. The 
program also produces a sequential list of decision points or waypoints to traverse in 
order to complete the course. Each leg of a participant’s route is compared to the LISP 
program route legs. If two LISP routes have legs that are the same, the leg is classified as 
the easier of the two routes. If a participant’s planned route between control points is not 
the same as any of the conqjuter program’s planned routes, the participant’s route is 
assigned a classification which is most closely associated with the participant’s route with 
respect to the program algorithm’s defining characteristics. 
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Figure L.5. LISP Beginner Route 




Figure L.6. LISP Intermediate Route 
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FcmI Ord Land Navigation Map 







Figure L.7. LISP Advanced Route 



5. PARTICIPANT ROUTE CLASSIFICATION 

The following are the results of the route classifications (Tables LA, L.5, and L.6) 
of each leg of each participant’s planned route and the overall route rating for each 
participant is based on MAJ Banker’s route classifications (Appendix L.2) and the LISP 
Program’s route classifications (Appendix L.3). Each leg was evaluated as Beginner (B), 
Intennediate (I) or Advanced (A). The summation of the routes were assessed by 
equating each leg classification with a numerical value (Beginner = 1, Intermediate = 2, 
and Advanced = 3) and summing the value of each leg. The number of legs on the course 
then divided this value. The resulting aggregate was then used to determine the difficulty 
level of the entire route. An “X” in the position of errors committed indicates that a 



participant did not attempt this leg of the route. 
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Table LA. Participant Route Classifications 
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Table L.5. Participant Route Classifications 
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Table L.6. Participant Route Classifications 



199 




6. ROUTE FROM CONTROL POINT 9 TO CONTROL POINT 4 

During the execution of the course, participants who made it to the end of the 
course. Control Point 9, were required to describe and execute a route from control Point 
9 to Control Point 4. Figure L.8 shows an example of the most efficient route from 
Control Point 9 to Control Point 4 for a beginning navigator. The program limitations 
allowed the route to run through a maximum of 10 decision points. 



Fort Otd Land Naviqation Map 




Figure L.8. LISP Beginner Route from CP 9 to CP 4 
Because of the limitations of the program, the route depicted is more difficult than 

the route chosen by any of the participants who performed this task. The LISP route does 

display characteristics of the routes chosen by the participants. The route travels major 

trails which have been traversed by the participant in the past. For example the route 

departs from Control Point Number 9 and heads south down the trail towards the five star 

intersection. The route then turns to the west and heads down the ridge towards the 

intersection south of Control Point 9. Compared to the route from Control Point 8 to 

Control Point 9, this is the same terrain covered by the Beginning and Intermediate 

navigators. 
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APPENDIX M. DATA COLLECTION WORKSHEETS 



1. TRAINING PHASE DATA COLLECTION SHEET 



PARTIOP,^^ ^ 


Session Date: 
Session Start Time: 
Session End Time: 




Initial Subject Study a) Study Map b) Read Map and Start Mvt c) Explore Terrain 


Method: 




1 






1 


Number Compass Checks: 




NA 


Number Map Checks: 




NA 


Number of times subject became "lost**: 


NA 


Number of times subject went out of bounds or fell off the edge of the model: NA 


Did the subject have difficulty reading the compass? 


Yes No NA 

1 1 


Did the subject have difficulty reading the map? 


Yes No NA 

1 1 


Did the subject have difficulty with the model interface? 


Yes No NA 

1 1 



Comments/Observations: 
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2. EVALUATION PHASE DATA COLLECTION SHEET 



PARTICIPANT 


Session Date: 
Session Start Time: 
Session End Time: 




RECORl 




Task f .. 


Mweti Lp 




■tiEIapsedfS 

pThnei^: 


Nimof 


NewRouta 


fPartidpant’s 
':’ Acti^'And* 
X Comroents 


bl^rvations^ 


1 
















2 


Move to CP #2 














^L3.Laix 


Indl^te Locatioo’l; 
Sr. 




Direction:? 


C^U^ 


Bearing:0 




Time: 'xx^fes ■ * - 




Indicate L<H'ati«^ 
CP'#5"^ 




Directtontil 


C<dor:si: 

filings: 


Rearing : 


«|f 


Q^n^tion: ^ 




indleate^^^oaS 




Diri^^[:t 


Cplo^^ 








3.2 


Move to CP #3 














4 


Move to CP #4 














5,I.a 

'■ , A 


indicate Locationll 
CP 


acNM^-. 


Direction:4\ 


C^ot;|| 


Bearii^'^^ 




Timei.^^^pftjx 


5,I.b 


indicate Locatlong 

CP #6 'tPSpS^' 


ipZAf*- 


Direction:-; 

,'' • 


Colon’-t 


Bearingj^^ 






5a.c 


indicateLocatiim' ; 




PlrectfaMBf 


'colon's 








5.2 


Move to CP #5 














6 


Move to CP #6 














7 


Move to CP #7 














8 


Move to CP #8 
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Move to CP #9 














Ip4 

10.it 


DirectkHi and 
Distance to CP#4 , 
fromCP#9 ■■ 


WA ,, 


,f - 


N/A 


N/A " 


GO/NGO 




' 1 ' 


Directions from ' 

CP#9toCP#4 

(Verbal) 




^N/A,'H!;, 


N/A 

'SH', 


'r-~N/A':^ 

■TSles vS^is 

\ \ V 


xTv.-.'V: 




10.3 


MovetoCP#4-_ 






v'v '. ' 


1^. "'1 




.X- xWHl^iS' ' ' ' , 


11 


White Board Test 


N/A 




N/A 






Order: 


Remarks: 
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APPENDIX N. PARTICIPANT DATA 



1. GENERAL 

Subject data consists of five items: Map with plaimed route, map with executed 
route, of Wheel Test at Control Point #2, of Wheel Test at Control Point #4, and of White 
Board Test. The errors for deviation from the planned route are located in Appendix O. 
The angle and distance measurements for the Wheel and White Board Tests can be found 
in Appendbc 0.2 and 0.3 respectively. 

The correct representations for the Wheel and White Board Tests are shown in 
Figures N.l, N.2, and N.5 respectively. Examples of digital photos of actual subject 
results for the three tests can be seen in Figures N.3, N.4, and N.6. Subject results for 
these tests will be displayed with the subject’s answers in solid lines or numbers 
superimposed over the actual answers which are displayed in dashed lines or shaded 
numbers. 




Figure N.l. Correct Wheel Test CP # 2 Figure N.2. Correct Wheel Test CP # 4 
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Figure N.3. Example Subject Wheel Test CP # 2 




Figure N.4. Example Subject Wheel Test CP # 4 
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Figure N.5. Correct White Board Test 




Figure N.6. Example Subject White Board Test 
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2. MAP PARTICIPANT NUMBER 1 




Figure N.7. Ml Planned Route 
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Figure N.8. Ml Executed Route 



207 






Figure N.9. Ml Wheel Test CP # 2 



Figure N. 10. Ml Wheel Test CP # 4 
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3. MAP PARTICIPANT NUMBER 2 




Figure N. 12. M2 Planned Route 
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Figure N. 13. M2 Executed Route 
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Figure N. 14. M2 Wheel Test CP # 2 




Figure N. 15. M2 Wheel Test CP # 4 
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4. MAP PARTICIPANT NUMBER 3 




Figure N.17. M3 Planned Route 
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Figure N.18. M3 Executed Route 
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Figure N. 19. M3 Wheel Test CP # 2 



Figure N.20. M3 Wheel Test CP # 4 
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5. MAP PARTICIPANT NUMBER 4 




Figure N.22. M4 Planned Route 
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Figure N.23. M4 Executed Route 
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Figure N.24. M4 Wheel Test CP # 2 



Figure N.25. M4 Wheel Test CP # 4 
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6. MAP PARTICIPANT NUMBER 5 




Figure N.27. M5 Planned Route 
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Figure N.28. M5 Executed Route 
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SUBJECT DID NOT 
REACH CP #4; 

NO DATA 
COLLECTED 



Figure N.29. M5 Wheel Test CP # 2 



Figure N.30. M5 Wheel Test CP # 4 




Figure N.31. M5 White Board Test 
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7. REAL WORLD PARTICIPANT NUMBER 1 
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Figure N.32. RW 1 Planned Route 
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Figure N.33. RWl Executed Route 
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Figure N.34. RWl Wheel Test CP # 2 



Figure N.35. RWl Wheel Test CP # 4 




Figure N.36. RWl White Board Test 
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8. REAL WORLD PARTICIPANT NUMBER 2 
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Figure N.37. RW2 Planned Route 
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Figure N.38. RW2 Executed Route 
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Figure N.40. RW2 Wheel Test CP # 4 




Figure N.4 1 . RW2 White Board Test 
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9. REAL WORLD PARTICIPANT NUMBER 3 




Figure N.42. RW3 Planned Route 
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Figure N.43. RW3 Executed Route 
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Figure N.44. RW3 Wheel Test CP # 2 



Figure N.45. RW3 Wheel Test CP # 4 




Figure N.46. RW3 White Board Test 
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10. REAL WORLD PARTICIPANT NUMBER 4 




i.' ! i 

lyiii 

■ iitUIIEI 

, J 

IS ill 



<i@o 






5 

a: 



III I 



s 

E 

=3 

z 



J I i 

O tf) 'r 



tf) , 



ir 



£ I 

1 ^ 

2 § 






•c 

TO 

w 




Figure N.47. RW4 Planned Route 
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Figure N.48. RW4 Executed Route 
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SUBJECT DID NOT 
REACH CP #4; 

NO DATA 
COLLECTED 



Figure N.50. RW4 Wheel Test CP # 4 




Figure N.5 1 . RW4 White Board Test 
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11. REAL WORLD PARTICIPANT NUMBER 5 
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Figure N.52. RW5 Planned Route 



233 





Figure N.53. RW5 Executed Route 



234 










Figure N.56. RW5 White Board Test 
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12. VIRTUAL ENVIRONMENT PARTICIPANT NUMBER 1 
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Figure N.57. VEl Planned Route 
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Figure N.58. VEl Executed Route 
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Ft Ord California 

Map by M a^r William P. Banker USA 
Field Checked by Captian Simon R. Goerger USA 
May 1990 







Figure N.61. VEl White Board Test 
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13. VIRTUAL ENVIRONMENT PARTICIPANT NUMBER 2 




Figure N.62. VE2 Planned Route 
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Figure N.63. VE2 Executed Route 
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Figure N.64. VE2 Wheel Test CP # 2 



Figure N.65. VE2 Wheel Test CP # 4 




Figure N.66. VE2 White Board Test 
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14. VIRTUAL ENVIRONMENT PARTICIPANT NUMBER 3 




Figure N.67. VE3 Planned Route 
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Figure N.68. VE3 Executed Route 
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Figure N.69. VE3 Wheel Test CP # 2 



Figure N.70. VE3 Wheel Test CP # 4 




Figure N.7 1. VE3 White Board Test 
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15. VIRTUAL ENVIRONMENT PARTICIPANT NUMBER 4 




Figure N.72. VE4 Planned Route 
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Figure N.73. VE4 Executed Route 
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Figure N.74. VE4 Wheel Test CP # 2 



Figure N.75. VE4 Wheel Test CP # 4 
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16. VIRTUAL ENVIRONMENT PARTICIPANT NUMBER 5 




Figure N.77. VE5 Planned Route 
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Figure N.78. VE5 Executed Route 
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SUBJECT DID NOT 
REACH CP #4; 

NO DATA 
COLLECTED 



Figure N.80. VE5 Wheel Test CP # 4 




Figure N. 81. VE5 White Board Test 
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APPENDIX O. RAW DATA 

1. GENERAL INFORMATION 

Participant data is referenced by the participant identification (ID) label (A - 
Assistant, M — Map, P - Pilot, RW - Real World, VE - Virtual Environment). The 
number corresponds to the participants internal group label. Data fields that are left 
blank represent information not recorded because a participant did not undergo the test or 
failed to reach that point in the course. Pilot participant data and assistant data is utilized 
for questionnaires only. The course data varied due to changes in experiment 
methodology for some of the pDot participants. Empty data fields represent information 
not recorded because the participant failed to undergo that portion of the experiment or 



did not wish to answer the question. 



Participant 

ID 


Group 


Age 


Sex 


Rank 


Service 


Branch 


Self Assessed 
Ability 


Test Date 


Test 

Time 


Ml 


Map 


29 


M 


03 


Army 


EN 


Intermediate 


17-May 


8:00 


M2 


Map 


30 


M 


03 


Army 


AR/CAV 


Expert 


1-Jun 


15:00 


M3 


Map 


33 


M 


03 


Marine 


AV 


Intermediate 


5-Jun 


7:30 


M4 


Map 


30 


M 


03 


Marine 


FA/MI 


Intermediate 


22-Jun 


12:00 


M5 


Map 


39 


M 


05 


Navy 


AV 


Intermediate 


21-Jul 


6:30 


RWl 


Real World 


34 


M 


04 


Army 


SC 


Intermediate 


29-May 


13:00 


RW2 


Real World 


29 


M 


03 


Navy 


SEAL 


Intermediate 


14-Jun 


13:00 


RW3 


Real World 


37 


M 


04 


Marine 


AV 


Intermediate 


16-Jun 


13:00 


RW4 


Real World 


30 


M 


03 


Marine 


AV 


Intermediate 


lO-Jul 


7:00 


RW5 


Real World 


34 


M 


03 


Army 


AV 


Intermediate 


18-Jul 


8:00 


VEl 


Virtual Env 


30 


M 


03 


Army 


AR/CAV 


Expert 


16-May 


8:00 


VE2 


Virtual Env 


28 


M 


03 


Marine 


IN 


Intermediate 


20-May 


8:00 


VE3 


Virtual Env 


34 


M 


NA 


Civilian 


CIV 


Intermediate 


1-Jun 


12:30 


VE4 


Virtual Env 


29 


F 


03 


Marine 


MI 


Beginner 


3-Jun 


12:30 


VE5 


Virtual Env 


35 


M 


04 


Army 


AV 


Beginner 


lO-Jul 


13:00 


A1 


Pilot Grp 1 


21 


M 


CDT 


Air Force 


Cadet 


Beginner 


18-May 


17:00 


PI 


Pilot Grp 1 


38 


M 


04 


Marine 


FA/MI 


Expert 


15-May 


8:00 


P2 


Pilot Grp 1 


34 


M 


04 


Marine 


IN 


Expert 


15-May 


13:00 


P3 


Pilot Grp 1 


28 


M 


03 


Marine 


IT 


Expert 


16-May 


13:00 


P4 


Pilot Grp 1 


39 


M 


04 


Marine 


UNK 


Intermediate 


17-May 


13:00 
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2. INITIAL TESTES AND QUESTIONNAIRE RESULTS 

The initial tests and questionnaires are in Appendix E. Answers for the Map Test 
are located in Appendix E. 1 . 



Participant 

ID 


Map 

Test 

Score 


Bar 

Evaluation 


Santa 
Barbara 
(Raw Score) 


Santa 

Barbara 

(Normalized) 


Santa 

Barbara 

(Group) 


GZ 

Score 


GZ 

Ability 

Group 


Banker 

Ability 

Group 


Ml 


19 


Intermediate 


42 


2.80 


High 


48.75 


High 


Beginner 


M2 


17 


Expert 


46 


3.07 


High 


22 


High 


Intermediate 


M3 


17 


Intermediate 


35 


2.33 


High 


37 


High 


Intermediate 


M4 


20 


Intermediate 


31 


2.07 


High 


12.25 


Low 


Intermediate 


M5 


16 


Intermediate 


36 


2.40 


High 


16.25 


Low 


Beginner 


RWl 


16 


Intermediate 


40 


2.67 


High 


10.25 


Low 


Beginner 


RW2 


19 


Intermediate 


34 


2.27 


High 


11.5 


Low 


Intermediate 


RW3 


18 


Intermediate 


35 


2.33 


High 


28.5 


High 


Intermediate 


RW4 


19 


Intermediate 


62 


4.13 


Low 


18.5 


Low 


Beginner 


RW5 


19.5 


Expert 


35 


2.33 . 


High 


21.25 


High 


Intermediate 


VEl 


18.5 


Expert 


31 


2.07 


High 


8.25 


Low 


Intermediate 


VE2 


19.5 


Intermediate 


39 


2.60 


High 


12.75 


Low 


Intermediate 


VE3 


15 


Intermediate 


30 


2.00 


High 


22.25 


High 


Intermediate 


VE4 


17 


Beginner 


45 


3.00 


High 


31.25 


High 


Beginner 


VE5 


13 


Beginner 


49 


3.27 


Low 


8.75 


Low 


Beginner 


A1 


14.5 


Beginner 


A1 


3.13 


High 


26.75 


High 


Beginner 


PI 


19.5 










1.75 


Low 




P2 


17 










19.25 


Low 




P3 


17.5 










24 


High 




P4 


18 










24.75 


High 





3. ROUTE ERRORS 

The data provided in this section consists of the map checks, errors, error 
distances, and route leg classifications. The data appears in its raw form, summations, 
and normalized form for each of the experiments fifteen participants. The codes for the 
utilized are listed in Table 0. 1 and in the List of Abbreviations (pp ). 



Abbreviation 


Category 


C-# 


Compass Check - Leg Number 


M-# 


Map Check - Leg Number 


MC-# 


Map and Compass Check - Leg Number 


MCL-# 


Map and Compass Check, Location Provided by Monitor - Leg Number 


OB-# 


Out of Bounds - Leg Number 


New Rt - # 


New Route Planned - Leg Number 



Table 0.1. Route Errors Abbreviation Table 
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a. 



Route Data Sununation 



Participant 

ID 


Controls 

Attempted 


Control 

Found 


Landmark 

Score 


(Banker) 

Average 

Planned 

Route 

Difficulty 

Level 


(Lisp) 

Average 

Planned 

Route 

Difficulty 

Level 


Ml 


9 


9 


9.00 


1.56 


1.67 


M2 


9 


9 


9.00 


1.56 


1.56 


M3 


9 


9 


9.00 


1.78 


1.89 


M4 


7 


6 


6.33 


2.33 


111 


M5 


4 


3 


3.66 


1.44 


1.67 


RWl 


6 


5 


5.33 


2.67 


i.fn 


RW2 


8 


8 


8.00 


2.44 


2.44 


RW3 


9 


9 


9.00 


1.44 


1.56 


RW4 


4 


3 


3.33 


2.67 


2.56 


RW5 


9 


9 


9.00 


1.11 


1.00 


VEl 


9 


9 


9.00 


1.67 


1.67 


VE2 


4 


3 


3.33 


1.44 


1.44 


VE3 


7 


7 


7.00 


1.56 


1.78 


VE4 


7 


6 


6.33 


1.22 


1.11 


VE5 


4 


3 


3.33 


2.00 


2.22 



Participant 

ID 


Errors- 

Tot 


Errors 
Per CP 
Attempt 


Dist-Tot 


Dist Per 
Error 


Normalized 

Error 

Score/Attempt 


Normalized 

Error 

Score/Found 
_ (All) 


Normalized 

Error 

Score/Found 

(Only) 


Ml 


5 


0.56 


3260 


652 


72.44 


72.44 


72.44 


M2 


5 


0.56 


937 


187.4 


20.82 


20.82 


20.82 


M3 


5 


0.56 


638 


127.6 


14.18 


14.18 


14.18 


M4 


5 


0.71 


3690 


738 


105.43 


123 


128.46 


M5 


4 


1.00 


3237 


809.25 


202.31 


269.75 


213.56 


RWl 


4 


0.67 


4017 


1004.25 


167.38 


200.85 


240.73 


RW2 


5 


0.63 


2136 


427.2 


53.4 


53.4 


53.4 


RW3 


3 


0.33 


448 


149.33 


16.59 


16.59 


16.59 


RW4 


7 


1.75 


4053 


579 


144.75 


193 


164.47 


RW5 


5 


0.56 


815 


163 


18.11 


18.11 


18.11 


VEl 


5 


0.56 


1270 


254 


28.22 


28.22 


28.22 


VE2 


5 


1.25 


6488 


1297.6 


324.4 


432.53 


442.22 


VE3 


7 


1.00 


3930 


561.43 


80.2 


80.2 


80.2 


VE4 


6 


0.86 


1593 


265.5 


37.93 


44.25 


47.1 


VE5 


6 


1.50 


4540 


756.67 


189.17 


252.22 


157.89 
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Participant 

ID 


Total 

Map 

Check 

Score 


Normalized 
Map Check 
Score 

(Attempted) 


Normalized 
Map Check 
Score 

(Found -All) 


Normalized 
Map Check 
Score 

(Found-Only) 


Controls 

Attempted 


Controls 

Found 


Landmark 

Score 


Ml 


10.5 


1.17 


1.17 


1.17 


9 


9 


9.00 


M2 


1.5 


0.17 


0.17 


0.17 


9 


9 


9.00 


M3 


13 


1.44 


1.44 


1.44 


9 


9 


9.00 


M4 


8.5 


1.21 


1.42 


1.42 


7 


6 


6.33 


M5 


12.50 


3.13 


4.17 


3 


4 


3 


3.66 


RWl 


34 


5.67 


6.8 


6.3 


6 


5 


5.33 


RW2 


23.5 


2.94 


2.94 


2.94 


8 


8 


8.00 


RW3 


0 


0 


0 


0 


9 


9 


9.00 


RW4 


16.5 


4.13 


5.5 


4.33 


4 


3 


3.33 


RW5 


13 


1.44 


1.44 


1.44 


9 


9 


9.00 


VEl 


0 


0 


0 


0 


9 


9 


9.00 


VE2 


31.5 


7.88 


10.5 


6.33 


4 


3 


3.33 


VE3 


22 


3.14 


3.14 


3.14 


7 


7 


7.00 


VE4 


14 


2 


2.33 


2.17 


7 


6 


6.33 


VE5 


10 


2.5 


3.33 


1.5 


4 


3 


3.33 



b. Route Data Leg SP to CPI 



Parti; .pant 
ID 


Errors 

P-I 


Total 

Dist-1 


C-1 


M-1 


MC-1 


MCL-1 


OB-1 


New Rt-1 


Checks 
Score SP-1 


Control 
Found SP-1 


Ml 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M2 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M3 


1 


168 


0 


0 


0 


0 


0 


0 


0 


1 


M4 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M5 


1 


1400 


0 


1 


0 


0 


0 


0 


1 


1 


RWl 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


RW2 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


RW3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


RW4 


1 


211 


0 


0 


0 


0 


0 


0 


0 


1 


RW5 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VEl 


1 


150 


0 


0 


0 


0 


0 


0 


0 


1 


VE2 


2 


3780 


0 


0 


3 


1 


4 


1 


16 


1 


VE3 


1 


230 


0 


0 


0 


0 


0 


0 


0 


1 


VE4 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VE5 


1 


263 


0 


0 


0 


0 


0 


0 


0 


1 
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c. Route Data Leg CPI to CP2 



Participant 

ID 


Errors 

7-2 


Total 

Dist-2 


C-2 


M-2 


MC-2 


MCL-2 


OB-2 


New Rt-2 


Checks 
Score 1-2 


Control 
Found 1-2 


Ml 


0 


0 


0 


0 


0 


0 


0 


0 


0.00 


1 


M2 


0 


0 


0 


0 


0 


0 


0 


0 


0.00 


1 


M3 


1 


216 


0 


0 


0 


0 


0 


0 


0.00 


1 


M4 


1 


176 


0 


0 


0 


0 


0 


0 


0.00 


1 


M5 


1 


150 


0 


1 


0 


0 


0 


0 


1.00 


1 


RWl 


1 


1233 


0 


1 


2 


0 


0 


0 


4.00 


1 


RW2 


0 


0 


0 


0 


0 


0 


0 


0 


0.00 


1 


RW3 


1 


133 


0 


0 


0 


0 


0 


0 


0.00 


1 


RW4 


3 


2129 


0 


2 


2 


1 


0 


1 


8.50 


1 


RW5 


1 


220 


0 


1 


0 


0 


0 


0 


1.00 


1 


VEl 


0 


0 


0 


0 


0 


0 


0 


0 


0.00 


1 


VE2 


0 


. 0 


0 


0 


.0 


0 


0 


0 


0.00 


1 


VE3 


0 


0 


0 


0 


0 


0 


0 


0 


0.00 


1 


VE4 


0 


0 


0 


0 


0 


0 


0 


0 


0.00 


1 


VE5 


1 


548 


0 


0 


0 


0 


0 


0 


0.00 


1 



d. Route Data Leg CP2 to CPS 



Participant 

ID 


Errors 

2-3 


Total 

Dist-3 


C-3 


M-3 


MC-3 


MCL-3 


OB-3 


New Rt-3 


Checks 
Score 2-3 


Control 
Found 2-3 


Ml 


1 


110 


0 


0 


0 


0 


0 


0 


0.00 


1 


M2 


1 • 


190 


0 


0 


0 


0 


0 


0 


0.00 


1 


M3 


0 


0 


0 


0 


0 


0 


0 


0 


0.00 


1 


M4 


2 


2077 


0 


1 


0 


1 


0 _ 


1 


4.50 


1 


M5 


1 


372 


0 


2 


0 


0 


1 


0 


4.00 


1 


RWl 


0 


0 


0 


0 


0 


0 


0 


0 


0.00 


1 


RW2 


0 


0 


0 


0 


0 


0 


0 


0 


0.00 


1 


RW3 


0 


0 


0 


0 


0 


0 


0 


0 


0.00 


1 


RW4 


1 


127 


0 


0 


0 


0 


0 


0 


0.00 


1 


RW5 


1 


227 


0 


1 


0 


0 


0 


0 


1.00 


1 


VEl 


1 


160 


0 


0 


0 


0 


0 


0 


0.00 


1 


VE2 


1 


200 


0 


0 


0 


0 


0 


0 


0.00 


1 


VE3 


1 


216 


0 


1 


0 


0 


0 


0 


1.00 


1 


VE4 


1 


180 


0 


0 


0 


0 


0 


0 


0.00 


1 


VE5 


1 


610 


0 


0 


0 


0 


0 


0 


0.00 


1 
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e. Route Data Leg CP3 to CP4 



Participant 

ID 


Errors 

3-4 


Total 

Dist~4 


C-4 


M-4 


MC-4 


MCL-4 


OB-4 


New Rt~4 


Checks 
Score 3-4 


Control 
Found 3-4 


Ml 


3 


3050 


0 


4 


0 


1 


0 


3 


8.5 


1 


M2 


1 


571 


0 


0 


1 


0 


0 


0 


1.5 


1 


M3 


1 


40 


0 


0 


0 


0 


0 


0 


0 


1 


M4 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M5 


1 


1315 


0 


3 


0 


0 


0 


1 


3.50 


1 


RWl 


2 


2378 


0 


3 


6 


1 


0 


1 


15.5 


1 


RW2 


2 


1174 


0 


2 


0 


1 


0 


1 


5.5 


1 


RW3 


2 


315 


0 


0 


0 


0 


0 


0 


0 


1 


RW4 


2 


1586 


0 


2 


1 


0 


0 


0 


3.5 


0 


RW5 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VEl 


1 


480 


0 


0 ■ 


0 


0 


0 


0 


0 


1 


VE2 


2 


2508 


0 


6 


2 


1 


0 


1 


12.5 


0 


VE3 


4 


3386 


1 


10 


1 


1 


0 


3 


17 


1 


VE4 


2 


1123 


0 


2 


0 


1 


0 


1 


5.5 


1 


VE5 


3 


3119 


0 


2 


0 


1 


0 


1 


5.5 


0 



f. Route Data Leg CP4 to CPS 



Participant 

ID 


Errors 

4-5 


Total 

Dist-5 


C-5 


M-5 


MC-5 


MCL-5 


OB-5 


New Rt-5 


Checks 
Score 4-5 


Control 
Found 4-5 


Ml 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M2 


1 


no 


0 


0 


0 


0 


0 


0 


0 


1 


M3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M4 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M5 






















RWl 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


RW2 


1 


275 


0 


1 


0 


0 


0 


0 


1 


1 


RW3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


RW4 






















RW5 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VEl 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VE2 






















VE3 


1 


98 


0 


0 


0 


0 


0 


0 


0 


1 


VE4 


1 


50 


0 


0 


0 


0 


0 


0 


0 


1 


VE5 
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g. Route Data Leg CPS to CP6 



Participant 

ID 


Errors 

5-6 


Total 

Dist-6 


C-6 


M-6 


MC-6 


MCL-6 


OB-6 


New Rt~6 


Checks 
Score 5-6 


Control 
Found 5-6 


Ml 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M2 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M3 


1 


54 


0 


0 


0 


0 


0 


0 


0 


1 


M4 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M5 






















RWl 


1 


406 


0 


1 


1 


0 


0 


0 


2.5 


0 


RW2 


1 


184 


0 


0 


0 


0 


0 


0 


0 


1 


RW3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


RW4 






















RW5 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VEl 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VE2 






















VE3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VE4 


1 


60 


0 


0 


0 


0 


0 


0 


0 


1 


VE5 























h. Route Data Leg CP6 to CP7 



Participant 

ID 


Errors 

6-7 


Total 

Dist-7 


C-7 


M-7 


MC-7 


MCL-7 


OB-7 


New Rt-7 


Checks 
Score 6-7 


Control 
Found 6-7 


Ml 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M2 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M3 


1 


160 


0 


1 


0 


0 


0 


0 


1 


1 


M4 


1 


607 


0 


2 


0 


0 


1 


0 


4 


0 


M5 






















RWl 






















RW2 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


RW3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


RW4 






















RW5 


1 


78 


0 


0 


0 


0 


0 


0 


0 


1 


VEl 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VE2 






















VE3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VE4 


1 


180 


0 


1 


0 


0 


0 


0 


1 


0 


VE5 
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i. Route Data Leg CP7 to CPS 



Participant 

ID 


Errors 

7-8 


Toted 

Dist-8 


C-8 


M-8 


MC-8 


MCL-8 


OB-8 


New Rt~8 


Checks 
Score 7^8 


Control 
Found 7-8 


Ml 


1 


100 


0 


0 


0 


0 


0 


0 


0 


1 


M2 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M4 






















M5 






















RWl 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


RW2 


1 


503 


0 


5 


0 


0 


0 


0 


5 


1 


RW3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


RW4 






















RW5 


1 


105 


0 


0 


0 


0 


0 


0 


0 


1 


VEl 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


VE2 






















VE3 






















VE4 






















VE5 























j. Route Data Leg CPS to CP9 



Participant 

ID 


Errors 

8-9 


Total 1 C-9 
Disi 


M-9 


MC-9 


MCL-9 


OB-9 


New Rt-9 


Checks 
Score 8-9 


Control 
Found 8-9 


Ml 


0 


0 0 


0 


0 


0 


0 


0 


0 


1 


M2 


7 


6( 


0 


0 


0 


0 


0 


0 


0 


1 


M3 


c 


0 


0 


0 


0 


0 


0 


0 


0 


1 


M4 






















M5 






















RWl 






















RW2 






















RW3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


RW4 






















RW5 


1 


185 


0 


0 


0 


0 


0 


0 


0 


1 


VEl 


2 


480 


0 


0 


0 


0 


0 


0 


0 


1 


VE2 




















VE3 




















VE4 






















VE5 
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k. Route Data Non Error Checks 



Participant 

ID 


C-Non 

Errors 


M-Non 

Errors 


MC-Non 

Errors 


MCL-Non 

Errors 


OB-Non 

Errors 


New Rt-Non 
Errors 


Checks Score 
- Non Errors 


Ml 


0 


2 


0 


0 


0 


0 


2 


M2 


0 


0 


0 


0 


0 


0 


0 


M3 


0 


11 


0 


0 


0 


2 


12 


M4 


0 


0 


0 


0 


0 


0 


0 


M5 


0 


0 


0 


0 


0 


0 


0 


RWl 


0 


0 


8 


0 


0 


0 


12 


RW2 


3 


0 


6 


0 


0 


0 


12 


RW3 


0 


0 


0 


0 


0 


0 


0 


RW4 


0 


0 


3 


0 


0 


0 


4.5 


RW5 


0 


8 


0 


0 


0 


4 


10 


VEl 


0 


0 


0 


0 


0 


0 


0 


VE2 


0 


2 


1 


0 


0 


0 


3.5 


VE3 


0 


2 


0 


0 


0 


1 


2.5 


VE4 


0 


6 


0 


0 


0 


1 


6.5 


VE5 


0 


4 


0 


0 


0 


0 


4 



1. Route Data Totals 



Participant 

ID 


Errors-Tot 


Distance 

■Tot 


Distance 
Per Error 


C-Tot 


M-Tot 


MC-Tot 


MCL-Tot 


OB-Tot 


New Rt 
■Tot 


Ml 


5 


3260 


652 


0 


6 


0 


1 


0 


3 


M2 


5 


937 


187.4 


0 


0 


1 


0 


0 


0 


M3 


5 


638 


127.6 


0 


12 


0 


0 


0 


2 


M4 


5 


3690 


738 


0 


3 


0 


1 


1 


1 


M5 


4 


3237 


809.25 


0 


10 


0 


0 


1 


1 


RWl 


4 


4017 


1004.25 


0 


5 


17 


1 


0 


1 


RW2 


5 


2136 


427.2 


3 


8 


6 


1 


0 


1 


RW3 


3 


448 


149.33 


0 


0 


0 


0 


0 


0 


RW4 


7 


4053 


579 


0 


4 


6 


1 


0 


1 


RW5 


5 


815 


163 


0 


11 


0 


0 


0 


4 


VEl 


5 


1270 


254 


0 


0 


0 


0 


0 


0 


VE2 


5 


6488 


1297.6 


0 


8 


6 


2 


4 


1 


VE3 


7 


3930 


561.43 


1 


13 


2 


1 


0 


4 


VE4 


6 


1593 


265.5 


0 


10 


0 


1 


0 


2 


VE5 


6 


4540 


756.67 


0 


6 


0 


1 


0 


2 
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m. Leg Difficulty Evaluation Banker 

Each leg is evaluated utilizing MAJ Banker’s Route Classification 
(Appendix L.2). “B” stands for Beginner, “I” stands or Intermediate, and “A” stands for 
Advanced. The total is based on a point value system of B = 1, 1 =2, and A = 3. The 
Average is the total divided by the number of legs. 0-1.50 is an average course difficulty 
of Beginner, 1.51-2.5 is Intermediate, and 2.51-3.0 is Advanced. 



Participant 

ID 


Legl 


Leg 2 


Leg 3 


Leg 4 


Legs 


Leg 6 


Leg? 


Legs 


Leg 9 


Total 


Average 


Ml 


B 


I 


I 


I 


I 


B 


I 


B 


B 


14 


1.56 


M2 


B 


I 


I 


I 


B 


B 


I 


I 


B 


14 


1.56 


M3 


I 


I 


I 


I 


I 


B 


I 


B 


I 


16 


1.78 


M4 


I 


I 


I 


A 


I 


B 


A 


A 


A 


21 


2.33 


M5 


B 


I 


I 


B 


I 


B 


I 


B 


B 


13 


1.4 


RWl 


I 


A 


I 


A 


I 


A 


A 


A 


A 


24 


2.67 


RW2 


I 


B 


I 


A 


I 


A 


A 


A 


A 


22 


2.44 


RW3 


B 


I 


I 


B 


I 


B 


I 


B 


B 


13 


1.44 


RW4 


A 


I 


I 


A 


I 


A 


A 


A 


A 


24 


:■ 67 


RW5 


B 


B 


I 


B 


B 


B 


B 


B 


B 


10 


1.11 


VEl 


B 


B 


I 


B 


B 


B 


A 


A 


I 


15 


1.67 


VE2 


B 


I 


I 


B 


B 


B 


B 


B 


A 


13 


1.44 


VE3 


B 


B 


I 


B 


B 


A 


B 


B 


A 


14 


1.56 


VE4 


B 


I 


B 


B 


B 


B 


I 


B 


B 


11 


1.22 


VE5 


A 


A 


B 


I 


B 


B 


A 


I 


A 


18 


2.00 



n. Leg Difficulty Evaluation LISP 

Each leg is evaluated utUizing LISP Programs Route Classification (Appendix 
L.3). See above for code definitions and summation specifics. 



Participant 

ID 


Legl 


Leg! 


Leg 3 


Leg 4 


Legs 


Leg 6 


Leg? 


Legs 


Leg 9 


Total 


Average 


Ml 


B 


B 


I 


I 


I 


1 


1 


1 


B 


15 


1.67 


M2 


B 


B 


I 


I 


B 


1 


1 


I 


B 


14 


1.56 


M3 


I 


B 


I 


I 


I 


I 


I 


I 


I 


17 


1.89 


M4 


I 


B 


I 


A 


I 


B 


A 


A 


A 


20 


2.22 


M5 


B 


B 


I 


B 


I 


B 


1 


1 


B 


15' 


1.7 


RWl 


I 


A 


I 


A 


I 


A 


A 


A 


A 


24 


2.67 


RW2 


1 


B 


I 


A 


I 


A 


A 


A 


A 


22 


2.44 


RW3 


B 


B 


I 


B 


I 


1 


1 


1 


B 


14 


1.56 


RW4 


A 


B 


I 


A 


I 


A 


A 


A 


A 


23 


2.56 


RW5 


B 


B 


B 


B 


B 


B 


B 


B 


B 


9 


1.00 


ve; 


B 


B 


I 


B 


B 


1 


1 


A 


I 


15 


1.67 


ve; 


B 


B 


I 


B 


B 


I 


B 


B 


A 


13 


1.44 


VE3 


B 


I 


I 


B 


B 


A 


B 


1 


A 


16 


1.78 


VE4 


B 


B 


B 


B 


B 


I 


B 


B 


B 


10 


1.11 


VE5 


A 


A 


B 


I 


B 


1 


A 


1 


A 


20 


2.22 
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4. WHEEL TEST RESULTS 



a. Wheel Test Results for Control Point 2 



Participant 

ID 


Orient 


Time (sec) 


SP 


CPS 


CP9 


DifCPl 


DifCPe 


DifCPS 


A ve Angular Diff CP2 


Ml 


North 


47 


124 


271 


164 


-54 


-31 


6 


30.33 


M2 


South 


12 


78 


229 


166 


-8 


11 


4 


7.67 


M3 


North 


16 


77 


248 


147 


-7 


-8 


23 


12.67 


M4 


West 


43 


41 


183 


120 


29 


57 


50 


45.33 


M5 


South 


40 


118 


201 


145 


-48 


39 


25 


37.33 


RWl 


South 


58 


76 


204 


113 


-6 


36 


57 


33.00 


RW2 


South 


17 


85 


240 


141 


-15 


0 


29 


14.67 


RW3 


South 


27 


65 


206 


167 


5 


34 


3 


14.00 


RW4 


West 


29 


52 


163 


88 


18 


77 


82 


59.00 


RW5 


North 


15 


68 


243 


122 


2 


-3 


48 


17.67 


VEl 


Arrows 


48 


83 


249 


166 


-13 


-9 


4 


8.67 


VE2 


South 


53 


86 


237 


129 


-16 


3 


41 


20.00 


VE3 


South 


15 


87 


242 


141 


-17 


-2 


29 


16.00 


VE4 


South 


40 


74 


218 


164 


-4 


22 


6 


10.67 


VE5 


SE 


22 


59 


161 


124 


11 


79 


46 


45.33 



b. Wheel Test Results for Control Point 4 and Total Wheel Test Angular Difference 
Total Wheel Test Angular Difference is the value of the Average Angular 
Differences for CP2 and CP4 divided by two. No data was collected on four individuals 
(M5, RW4, VE2, and VE5) because they failed to reach Control Point 4. Their Total 
Wheel Test Angular Difference is the save as the value for the Average Angular 
Differences for CP2. 



Participant 

ID 


Orient 


Time 

(sec) 


CPI 


CP6 


CPS 


Diff 

CPI 


Diff 

CP6 


Diff 

CPS 


Ave Angular 
DiffCP4 


Total Wheel 
Test Angular 
Diff . .. 


Ml 


South 


38 


86 


227 


153 


-28 


8 


-37 


24.33 


27.33 


M2 


South 


16 


59 


234 


185 


-1 


1 


-69 


23.67 


15.67 


M3 


North 


18 


69 


252 


168 


-11 


-17 


-52 


26.67 


19.67 


M4 


South 


47 


49 


187 


110 


9 


48 


6 


21.00 


33.17 


M5 




















37.33 


RWl 


South 


28 


74 


230 


182 


-16 


5 


-66 


29.00 


31.00 


RW2 


South 


24 


57 


240 


180 


1 


-5 


-64 


23.33 


19.00 


RW3 


South 


27 


91 


247 


114 


-33 


-12 


2 


15.67 


14.83 


RW4 




















59.00 


RW5 


East 


29 


53 


221 


126 


5 


14 


-10 


9.67 


13.67 


VEl 


Arrows 


62 


30 


168 


110 


28 


67 


6 


33.67 


21.17 


VE2 




















20.00 


VE3 


East 


17 


71 


226 


151 


-13 


9 


-35 


19.00 


17.50 


VE4 


East/N 


60 


89 


269 


180 


-31 


-34 


-64 


43.00 


26.83 


VE5 




















45.33 
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5. WH 



BOARD RESULTS 

White Board Normalized Distance Differences from Actual Normalized 
Distances 



a. 



ID 


SPto 

CPI 


CPI to 
CP2 


CP2to 

CP3 


CP3to 

CP4 


CP 4 to 
CPS 


CPS to 
CP6 


CP6to 

CP7 


CP7to 

CPS 


CPS to 
CP9 


CP9to 

SP 


Total 

WB 

Norm 

Dist 


Avg 

WB 

Norm 

Dist 


Ml 


-0.01 


-0.01 


-0.01 


0.03 


0.03 


0.06 


-0.04 


-0.01 


0.03 


-0.08 


0.33 


0.033 


M2 


-0.01 


0.00 


-0.02 


-0.04 


0.06 


0.01 


-0.02 


-0.01 


0.04 


0.00 


0.23 


0.023 


M3 


0.02 


0.00 


-0.01 


-0.04 


0.03 


0.02 


0.00 


0.06 


-0.06 


-0.01 


0.27 


0.027 


M4 


0.00 


0.03 


-0.01 


-0.03 


0.02 


0.09 


-0.01 


-0.05 


-0.04 


0.00 


0.28 


0.028 


M5 


0.02 


0.00 


-0.03 


0.02 


0.03 


0.02 


-0.01 


0.01 


-0.03 


-0.03 


0.19 


0.019 


RWl 


0.00 


0.03 


0.00 


-0.05 


0.01 


0.04 


-0.02 


0.03 


-0.04 


-0.01 


0.22 


0.022 


RW2 


-0.01 


0.01 


0.01 


-0.03 


0.05 


0.03 


-0.02 


-0.01 


0.00 


-0.03 


0.19 


0.019 


RW3 


-0.03 


-0.01 


-0.01 


0.00 


0.02 


0.04 


0.02 


0.04 


-0.06 


0.00 


0.24 


0.024 


RW4 


-0.01 


0.01 


0.01 


-0.03 


0.05 


0.06 


-0.04 


-0.03 


-0.04 


0.02 


0.30 


0.030 


RW5 


-0.02 


-0.04 


0.01 


0.00 


0.01 


0.08 


-0.01 


0.00 


-0.03 


0.00 


0.20 


0.020 


VEl 


-0.01 


0.00 


0.01 


-0.03 


0.03 


0.02 


0.02 


0.03 


-0.05 


0.00 


0.20 


0.020 


VE2 


0.01 


0.00 


-0.01 


-0.02 


0.03 


0.02 


0.01 


0.03 


-0.02 


-0.04 


0.18 


0.018 


VE3 


-0.01 


0.00 


0.00 


0.00 


0.03 


0.03 


0.03 


0.01 


-0.07 


-0.03 


0.22 


0.022 


VE4 


-0.03 


0.00 


0.00 


-0.03 


0.03 


0.04 


0.07 


-0.01 


-0.04 


-0.04 


0.30 


0.030 


VE5 


-0.02 


0.01 


0.03 


-0.04 


0.01 


0.03 


-0.02 


0.01 


-0.03 


0.02 


0.23 


0.023 



b. White Board Angles 



ID 


SP,1,2 


1,2,3 


2,3,4 


3,4,S 


4,S,6 


S,6,7 


6,7,8 


7,8,9 


8,9,SP 


9,SP,1 


Ml 


161.23 


130.76 


162.90 


96.53 


47.81 


112.71 


112.29 


169.01 


177.02 


37.31 


M2 


133.81 


131.06 


173.55 


137.39 


101.58 


110.53 


92.07 


141.61 


121,66 


29.47 


M3 


125.11 


141.25 


130.59 


96.82 


96.61 


109.81 


90.87 


169.30 


166.80 


15.28 


M4 


85.52 


114.74 


158.31 


104.45 


62.82 


86.72 


165.41 


1 19.66 


149.12 


8.58 


M5 


123.36 


165.73 


169.87 


144.34 


89.52 


88.99 


162.31 


173.04 


126.86 


82.20 


RWl 


165.20 


151.99 


138.50 


90.27 


86.18 


100.58 


97.92 


145.11 


151.48 


33.95 


RW2 


178.66 


178.82 


122.12 


111.69 


90.88 


99.38 


120.00 


149.72 


140.01 


48.87 


RW3 


179.06 


120.49 


116.90 


105.25 


70.42 


102.98 


89.88 


84.67 


80.07 


31.09 


RW4 


173.85 


126.46 


157.38 


144.36 


98.15 


76.53 


157.19 


138.46 


136.72 


64.26 


RW5 


125.65 


139.22 


166.89 


83.93 


55.93 


82.50 


113.71 


145.63 


127.68 


29.28 


VEl 


137.66 


145.01 


158.50 


120.34 


82.54 


125.84 


78.04 


144.58 


152.02 


20.63 


VE2 


166.37 


150.42 


123.21 


86.74 


114.92 


92.34 


96.62 


179.27 


174.90 


39.97 


VE3 


119.82 


144.85 


131.46 


69.32 


59.17 


99.73 


97.81 


148.98 


142.00 


23.10 


VE4 


178.52 


145.75 


157.28 


116.17 


89.60 


90.86 


91.11 


1 12.73 


120.89 


74.89 


VE5 


119.67 


144.39 


144.32 


106.83 


103.46 


104.45 


96.59 


160.99 


145.83 


8.46 
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c. White Board Angles Differences from Actual Angles and Totals 



ID 


SP,1,2 


1,2,3 


2,3,4 


3,4,5 


4,5,6 


5,6,7 


6,7,8 


7,8,9 


8,9,SP 


9,SP,1 


Total 

WB 

Angle 

Diff 


Avg 

WB 

Angle 

Diff 


Ml 


27.64 


8.09 


-7.91 


-31.71 


-50.78 


6.29 


39.74 


22.39 


9.04 


26.26 


229.87 


22.987 


M2 


0.23 


8.39 


2.74 


9.15 


2.99 


4.11 


19.52 


-5.01 


-46.32 


18.42 


116.87 


11.687 


M3 


-8.48 


18.58 


-40.22 


-31.42 


-1.98 


3.39 


18.32 


22.68 


-1.17 


4.24 


150.47 


15.047 


M4 


-48.07 


-7.93 


-12.50 


-23.79 


-35.77 


-19.69 


92.86 


-26.96 


-18.86 


-2.47 


288.88 


28.888 


M5 


-10.22 


43.07 


-0.94 


16.10 


-9.08 


-17.43 


89.75 


26.42 


-41.11 


71.16 


325.27 


32.527 


RWl 


31.62 


29.32 


-32.32 


-37.97 


-12.41 


-5.84 


25.37 


-1.51 


-16.49 


22.90 


215.74 


21.574 


RW2 


45.07 


56.15 


-48.70 


-16.55 


-7.72 


-7.04 


47.45 


3.10 


-27.97 


37.82 


297.57 


29.757 


RW3 


45.48 


-2.18 


-53.91 


-22.99 


-28.17 


-3.44 


17.33 


-61.95 


-87.90 


20.04 


343.39 


34.339 


RW4 


40.27 


3.79 


-13.43 


16.13 


-0.45 


-29.89 


84.64 


-8.16 


-31.25 


53.21 


281.21 


28.121 


RW5 


-7.94 


16.55 


-3.92 


-44.31 


-42.66 


-23.92 


41.16 


-0.99 


-40.30 


18.23 


239.97 


23.997 


VEl 


4.07 


22.34 


-12.31 


-7.90 


-16.05 


19.42 


5.49 


-2.04 


-15.96 


9.58 


115.15 


11.515 


VE2 


32.78 


27.75 


-47.60 


-41.50 


16.33 


-14.08 


24.07 


32.65 


6.92 


28.93 


272.62 


27.262 


VE3 


-13.77 


22.18 


-39.35 


-58.91 


-39.42 


-6.69 


25.25 


2.36 


-25.98 


12.06 


245.97 


24.597 


VE4 


44.94 


23.08 


-13.53 


-12.07 


-8.99 


-15.56 


18.55 


-33.89 


-47.08 


63.84 


281.54 


28.154 


VE5 


-13.91 


21.72 


-26.49 


-21.41 


4.87 


-1.97 


24.04 


14.37 


-22.15 


-2.59 


153.51 


15.351 



6. UNPLANNED ROUTE RESULTS 



Participant 

ID 


Direction from 
CP9toCP4 


Distance 
to CP 4 


Route 


Time 

(min) 


Errors 

Unplanned Route 


Distance 
Unplanned Route 


Ml 


West 


200m 


Trail S & E to 
CP8, WtoCP4 


6:17 


0 


0 


M2 


West 


200m 


Trail N to 
clearing, W to 
CP4 


6:07 


0 


0 


M3 


WestSouthWest 


350m 


Trail toward CP8, 
trail toward CP6, 
& E to CP4 


7:26 


0 


0 


M4 














M5 














RWl 














RW2 














RW3 


WestSouthWest 


300m 


Trail N to CP3,W 
toCP4 


3:28 


0 


0 


RW4 














RW5 


WestNorthWest 


250m 


Trail N to CP3,W 
toCP4 


4:53 


0 


0 


VEl 


West 


400m 


Trail N to CP3.W 
toCP4 


4:39 


0 


0 


VE2 






' 








VE3 














VE4 














VE5 
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7. DEBRIEFmG QUESTIONNAIRE RESULTS 
a. Map Questions 



Participant 

ID 


Question 1 


Question 2 


Question 3 


Question 4 


Question 5 


Question 6 


Question 7 


Ml 


2 


4 


4 


5 


5 


4 


2 


M2 


3 


4 


4 


5 


3 


4 


2 


M3 


4 


4 


5 


5 


4 


4 


2 


M4 


4 


5 


5 


5 


5 


5 


1 


M5 


4 


3 


3 


4 


3 


4 


2 


RWl 


4 


4 


2 


2 


3 


3 


3 


RW2 


4 


4 


3 


4 


4 


4 


2 


RW3 


4 


3 


5 


5 


4 


3 


1 


RW4 


2 


2 


3 


2 


2 


3 


3 


RW5 


4 


4 


4 


5 


4 


3 


2 


VEl 


5 


5 


4 


4 


4 


1 


2 


VE2 


4 


4 


2 


4 


3 


3 


2 


VE3 


3 


4 


3 


4 


4 


4 


1 


VE4 


4 


4 


4 


5 


3 


2 


2 


VE5 


4 


3 


4 


4 


3 


4 


2 


A1 


4 


5 


4 


3 


2 


2 


2 


PI 


5 


4 


3 


4 




3 


1 


P2 


5 


5 


3 


4 


4 


2 


1 


P3 


3 


5 


2 


4 


4 


2 


2 


P4 


5 


5 










1 



b. Course Questions 



Participant 

ID 


Question 1 


Question 2 


Question 3 


Question 4 


Question 5 


Ml 


2 


5 


4 


2 


4 


M2 


2 


4 


4 


4 


2 


M3 


3 


4 


5 


3 


4 


M4 


4 


5 


5 


4 


4 


M5 


5 


5 


4 


1 


5 


RWl 


4 


5 


4 


1 


5 


RW2 


3 


4 


4 


2 


3 


RW3 


2 


5 


5 


2 


2 


RW4 


4 


4 


2 


4 


5 


RW5 


3 


5 


5 


1 


4 


VEl 


2 


3 


4 


1 


2 


VE2 


4 


4 


4 


2 


3 


VE3 


3 


5 


5 


4 


2 


VE4 


5 


4 


4 


3 


5 


VE5 


5 


5 


5 


3 


5 


A1 


3 


5 


5 


3 


3 


PI 


3 


1 


5 


1 


5 


P2 


3 


5 


2 


3 


2 


P3 


3 


5 


5 


3 


4 


P4 
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c. Miscellaneous Questions 



Participant 

ID 


Question 1 


Question 2 


Question 3 


Question 4 


Question 5 


Ml 


5 


5 


2 


3 


5 


M2 


5 


4 


2 


4 


4 


M3 


5 


5 


1 


4 


4 


M4 


5 


5 


1 


5 


3 


M5 


5 


5 


1 


4 


3 


RWl 


4 


3 


3 


4 


1 


RW2 


4 


4 


2 


4 


1 


RW3 


5 


5 


1 


5 


4 


RW4 


5 


2 


5 


5 


2 


RW5 


5 


3 


3 


4 


4 


VEl 


5 


5 


1 


5 


5 


VE2 


5 


2 


4 


2 


3 


VE3 


5 


5 


1 


4 


4 


VE4 


5 


4 


2 


2 


1 


VE5 


5 


2 


4 


3 


2 


A1 


4 


2 


4 


5 


3 


PI 


5 


4 


2 


3 


2 


P2 


4 


4 


2 


4 


1 


P3 


5 


2 


4 


4 


2 


P4 


4 


5 


1 




2 



d. Model Questions 



/D 


Quest 

1 


Quest 

2 


Quest 

3 


Quest 

4 


Quest 

5 


Quest 

6 


Quest 

7 


Quest 

8 


Quest 

9 


Quest 

10 


Quest 

11 


Quest 

12 


Quest 

13 


Quest 

14 


Ml 






























M2 






























M3 






























M4 






























M5 






























RWl 






























RW2 






























RW3 






























RW4 






























RW5 






























VEl 


5 


5 


4 


4 


4 


1 


5 


3 


5 


1 


5 


5 


5 


5 


VE2 


4 


4 


2 


4 


3 


3 


3 


4 


4 


1 


4 


4 


4 


2 


VE3 


4 


4 


3 


5 


4 


4 


5 


4 


4 


2 


5 


5 


5 


5 


VE4 


5 


5 


4 


3 


3 


2 


2 


1 


4 


3 


4 


5 


5 


5 


VE5 


4 


4 


4 


4 


4 


4 


4 


2 


2 


4 


1 


1 


5 


1 


A1 


3 


4 


2 


5 


3 


3 


4 


4 


5 


2 


5 


5 


5 


5 


PI 


5 


5 


5 


5 


3 


4 


1 


1 


5 


2 


2 


1 


5 




P2 


4 


4 


4 


4 


4 


2 


2 


3 


4 


1 


4 


5 


5 


5 


P3 


4 


4 


4 


4 


3 


1 


3 


4 


4 


4 


4 


4 


4 


4 


P4 


3 


4 
















3 


3 


2 


3 


4 
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e. Interface Questions 



Participant 

ID 


Question 1 


Question 2 


Question 3 


Question 4 


Question 5 


Question 6 


Question 7 


Question 8 


Ml 


















M2 


















M3 


















M4 


















M5 


















RWl 


















RW2 


















RW3 


















RW4 


















RW5 


















VEl 


5 


4 


4 


4 


4 


5 


5 


1 


VE2 


4 


4 


4 


3 


4 


3 


3 


2 


VE3 


5 


5 


5 


5 


5 


5 


5 


1 


VE4 


5 


5 


5 


5 


5 


5 


5 


1 


VE5 


5 


5 


5 


5 


5 


5 


5 


1 


A1 


4 


5 


4 


5 


4 


5 


5 


1 


PI 


5 


5 


5 


5 


5 


5 


5 


2 


P2 


5 


5 


4 


4 


4 


5 


4 


2 


P3 


4 


5 


5 


5 


4 


4 


4 


3 


P4 


4 


4 


4 


4 


4 


5 


5 
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f. Descriptive Statistics for Debriefing Questionnaire 

The following table displays summary of the descriptive statistics for the 
questionnaire minus the data for the pilot participants. The total number of participants is 
15. Missing data is from questions that were not administered to real world and map 
group participants. 





Mean 


Std. Dev. 


Std. Error 


Count 


Minimum 


Maximum 


# Missing 


MapQl 


3.667 


.816 


.211 


15 


2.000 


5.000 


0 


MapQ2 


3.SOO 


.115 


.200 


15 


2.000 


5.000 


0 


MapQ3 


3.667 


.976 


.2S2 


15 


2.000 


5.000 


0 


MapQ4 


4.200 


1.014 


.262 


15 


2.000 


5.000 


0 


MapQ5 


3.600 


.828 


.214 


15 


2.000 


5.000 


0 


MapQ6 


3.400 


.986 


.2S4 


15 


1.000 


5.000 


0 


MapQ7 


1.933 


.S94 


.1S3 


15 


1.000 


3.000 


0 


CourseQl 


3.400 


1.121 


.289 


15 


2-000 


5.000 


0 


CourseQ2 


4.467 


.640 


.165 


15 


3.000 


5.000 


0 


CourseQS 


4.267 


.799 


.206 


15 


2.000 


5.000 


0 


CourseQ4 


2.467 


1.187 


.307 


15 


1.000 


4.000 


0 


CourseQS 


3.667 


1.234 


.319 


15 


2.000 


5.000 


0 


MiscQl 


4.S67 


.3S2 


.091 


15 


4.000 


5.000 


0 


MiscQ2 


3.933 


1.223 


.316 


15 


2.000 


5.000 


0 


MiscQS 


2.200 


1.320 


.341 


15 


1.000 


5.000 


0 


MiscQ4 . 


3.867 


.990 


.256 


15 


2.000 


5.000 


0 


MiscQS 


3.067 


1.387 


.358 


15 


1.000 


5.000 


0 


ModelQl 


4.400 


.S48 


.245 


5 


4.000 


5.000 


10 


ModelQ2 


4.400 


.S48 


.245 


5 


4.000 


5.000 


10 


ModelQ3 


3.400 


.894 


.400 


5 


2.000 


1.000 


10 


ModelQ4 


4.000 


.707 


.316 


5 


3.000 


5.000 


10 


ModelQS 


^ 3.600 


.S48 


.245 


5 


3.000 


1.000 


10 


ModelQ6 


1 2.800 


1.304 


.583 


5 


1.000 


1.000 


10 


ModelQ? 


3.800 


1.304 


.583 


5 


2.000 


5.000 


10 


ModelQS 


2.800 


1.304 


.583 


5 


1.000 


1.000 


10 


ModelQ9 


3.800 


1.09S 


.490 


5 


2.000 


5.000 


10 


ModelQlO 


2.200 


1.304 


.583 


5 


1.000 


1.000 


10 


ModelQ 11 i 


1 3.800 


1.643 


.735 


5 


1.000 


5.000 


10 


ModelQ12 | 


1 4.000 


1.732 


.775 


5 


1.000 


5.000 


10 


ModelQ 13 


j 4.800 


.447 


.200 


5 


1.000 


5.000 


10 


ModelQM 


3.600 


1.949 


.872 


5 


1.000 


5.000 


10 


InterfaceQl 


4.800 


.447 


.200 


5 


1.000 


5.000 


10 


InterfaceQ2 


4.000 


.S48 


.245 


5 


1.000 


5.000 


10 


lnterfaceQ3 


4.000 


.S48 


.245 


5 


1.000 


5.000 


10 


InterfaceQ4 


' 4.400 


.894 


.400 


5 


3.000 


5.000 


10 


InterfaceQS i 


4.000 


.S48 


.245 


5 


4.000 


5.000 


10 


InterfaceQ6 


4.600 


,894 


.400 


5 


3.000 


5.000 


10' 


InterfaceQ? 


4.600 


.894 


.400 


5 


3.000 


5.000 


10 


InterfaceQS 


1.200 


.447 


.200 


5 


1.000 


2.000 


10 



267 



g. Model Needs 

For the each item “1” means the items does need to be included and “0” means 
the items does not need to be included in a model used to prepare an individual to 
navigate through the actual terrain. 

1) Buildings 



Participant 

ID 


factory 


houses 


public buildings 


shacks 


Ml 


1 


1 


1 


1 


M2 


1 


1 


1 


1 


M3 


1 


1 


1 


1 


M4 


1 


1 


1 


0 


M5 


1 


1 


1 


1 


RWl 


1 


1 


1 


1 


RW2 


1 


1 


1 


1 


RW3 


1 


1 


1 


1 


RW4 


1 


1 


1 


1 


RW5 


1 


1 


1 


1 


VEl 


1 


1 


1 


1 


VE2 


1 


1 


1 1 


VE3 


1 


0 


1 


0 


VE4 


1 


1 


1 


0 


VE5 


1 


1 


1 


1 


A1 


1 


1 


1 


1 


PI 


1 


1 


1 


1 


P2 


1 


1 


1 


1 


P3 


1 


1 


1 


1 


P4 


1 


1 


1 


0 
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2) Miscellaneous Objects 



Participant 

ID 


animals 


compass 


people 


road signs 


rock piles 


sand bags 


sound 


street signs 


the sun 


Ml 


0 


1 


0 


0 


0 


0 


1 


0 


0 


M2 


0 


1 


0 


1 


0 


0 


0 


1 


0 


M3 


0 


1 


0 


0 


0 


0 


1 


1 


1 


M4 


0 


0 


0 


1 


1 


1 


0 


1 


1 


M5 


0 


1 


0 


1 


0 


0 


0 


1 


0 


RWl 


0 


1 


0 


1 


0 


1 


0 


0 


0 


RW2 


0 


1 


0 


0 


0 


0 


0 


0 


0 


RW3 


0 


0 


0 


0 


1 


0 


0 


0 


0 


RW4 


0 


1 


0 


0 


0 


0 


0 


0 


0 


RW5 


0 


1 


0 


0 


0 


0 


0 


1 


1 


VEl 


0 


1 


0 


0 


1 


0 


1 


0 


1 


VE2 


0 


1 


0 


0 


0 


0 


0 


0 


0 


VE3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


VE4 


0 


0 


0 


0 


0 


0 


0 


0 


0 


VE5 


0 


1 


0 


0 


1 


0 


0 


0 


1 


A1 


0 


1 


0 


1 


1 


1 


1 


1 


0 


PI 


0 


0 


0 


1 


0 


0 


0 


1 


1 


P2 


0 


1 


1 


1 


1 


1 


0 


1 


0 


P3 


0 


1 


0 


1 


0 


0 


1 


1 


1 


P4 


0 


1 


0 


0 


0 


0 


0 


0 


1 



3) Obstacles 



Participant 

ID 


electric lines 


pits/fox holes 


shallow ditches 


telephone poles 


towers 


trenches 


Ml 


1 


0 


0 


1 


1 


0 


M2 


1 


1 


1 


1 


1 


1 


M3 


0 


1 


1 


1 


1 


1 


M4 


1 


1 


1 


1 


1 


1 


M5 


1 


0 


0 


1 


1 


1 


RWl 


0 


0 


0 


1 


1 


1 


RW2 


1 


0 


0 


1 


1 


0 


RW3 


1 


1 


1 


1 


1 


1 


RW4 


1 


1 


0 


1 


1 


1 


RW5 


1 


0 


0 


1 


1 


1 


VEl 


1 


0 


1 


1 


1 


1 


VE2 


1 


0 


0 


1 


1 


0 


VE3 


1 


1 


1 


0 


1 


1 


VE4 


1 


0 


0 


1 


1 


0 


VE5 


1 


1 


1 


1 


1 


1 
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Participant 

ID 


electric lines 


pits/fox holes 


shallow ditches 


telephone poles 


towers 


trenches 


A1 


1 


1 


1 


1 


1 


1 


PI 


1 


0 


0 


1 


1 


0 


P2 


1 


0 


1 


1 


1 


1 


P3 


1 


1 


1 


1 


1 


1 


P4 


1 


0 


0 


1 


1 


1 



4) Roads 



Participant ID 


dirt roads 


foot paths 


paved roads 


trails 


Ml 


1 


0 


1 


1 


M2 


I 


0 


1 


1 


M3 


1 


1 


1 


1 


M4 


1 


1 


1 


1 


M5 


1 


1 


1 


1 


RWl 


1 


0 


1 


1 


RW2 


1 


0 


1 


0 


RW3 


1 


1 


1 


1 


RW4 


1 


0 


1 


1 


RW5 


1 


1 


1 


1 


VEl 


1 


0 


1 


1 


VE2 


1 i 


: 0 


1 0 


VE3 


1 


1 


1 


1 


VE4 


0 


0 


0 


0 


VE5 


1 


1 


1 


1 


A1 


1 


1 


1 


1 


PI 


1 


1 


1 


1 


P2 


1 


1 


1 


1 


P3 


1 


1 


1 


1 


P4 


1 


0 


1 


1 



5) Terrain 



Participant ID 


depressions 


draws 


hills 


knolls 


ridgelines 


spurs/fingers 


Ml 


1 


0 


1 


1 


1 


1 


M2 


1 


0 


1 


1 


1 


1 


M3 


1 


0 


1 


1 


1 


1 


M4 


1 


0 


1 


1 


1 


1 


M5 


1 


1 


1 


1 


1 


1 


RWl 


1 


0 


1 


0 


1 


1 


RW2 


0 


0 


1 


1 


1 


1 


RW3 


1 


0 


1 


1 


1 


1 


RW4 


1 


0 


1 


1 


1 


1 


RW5 


1 


0 


1 


1 


1 


1 


VEl 


1 


1 


1 


1 


1 


1 


VE2 


1 


0 


1 


0 


1 


1 


VE3 


1 


0 


1 


1 


1 


1 
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Participant ID 


depressions 


draws 


hills 


knolls 


rid^etines 


spurs/fin^ers 


VE4 


1 


0 


1 


0 


1 


1 


VE5 


1 


0 


1 


1 


1 


1 


A1 


1 


0 


1 


1 


1 


1 


PI 


1 


0 


1 


1 


1 


1 


P2 


1 


0 


1 


1 


1 


1 


P3 


1 


0 


1 


1 


1 


1 


P4 


1 


0 


1 


1 


1 


1 



6) Vegetation 



Participant ID 


bushes 


clearings 


flowers 


grass/weeds 


trees 


undergrowth 


Ml 


1 


1 


0 


0 


1 


0 


M2 


1 


1 


0 


1 


1 


0 


M3 


1 


1 


0 


0 


1 


0 


M4 


1 


1 


0 


1 


1 


0 


M5 


0 


1 


0 


0 


1 


0 


RWl 


1 


1 


0 


1 


1 


0 


RW2 


0 


1 


0 


0 


1 


0 


RW3 


0 


1 


0 


1 


1 


0 


RW4 


1 


1 


0 


0 


0 


0 


RW5 


1 


1 


0 


1 


1 


0 


VEl 


1 


1 


0 


0 


1 


0 


VE2 


0 


1 


0 


0 


1 


0 


VE3 


0 


1 


0 


0 


0 


0 


VE4 


0 


1 


0 


0 


0 


0 


VE5 


1 


1 


0 


1 


1 


0 


A1 


1 


1 


0 


1 


1 


0 


PI 


1 


1 


0 


1 


1 


1 


P2 


1 


1 


0 


1 


1 


0 


P3 


1 


1 


0 


1 


1 


0 


P4 


0 


1 


0 


0 


1 


0 



7) Miscellaneous Items 



Participant ID 


lakes 


marsh lands 


ponds 


puddles 


stream/river 


swamps 


Ml 


1 


1 


1 


0 


1 


1 


M2 


1 


1 


1 


0 


1 


1 


M3 


1 


1 


1 


0 


0 


1 


M4 


1 


1 


1 


0 


0 


1 


M5 


1 


1 


1 


0 


1 


1 


RWl 


1 


1 


1 


1 


0 


1 


RW2 


1 


1 


0 


0 


0 


0 


RW3 


1 


1 


1 


0 


0 


1 


RW4 


1 


0 


1 


0 


0 


0 


RW5 


1 


1 


1 


0 


0 


1 


VEl 


1 


1 


1 


1 


0 


1 


VE2 


1 


0 


1 


0 


0 


0 
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Participant ID 


lakes 


marsh lands 


ponds 


puddles 


stream/river 


swamps 


VE3 


1 


1 


1 


0 


0 


1 


VE4 


1 


1 


0 


0 


0 


1 


VE5 


1 


1 


1 


1 


0 


1 


A1 


0 


0 


0 


0 


0 


0 


PI 


1 


0 


1 


0 


1 


1 


P2 


1 


1 


1 


0 


0 


1 


P3 


1 


1 


1 


0 


1 


1 


P4 


1 


0 


1 


0 


0 


0 



h. Top Six Model Needs 



Participant 

ID 


1 


2 


3 


4 


5 


6 


Ml 


Hills 


Ridge Lines 


Dirt Roads 


Spurs 


Rivers/Streams 


Electric Lines 


M2 


Buildings 


Trails 


Electric Lines 


Trees 


Rivers/Streams 


Elevation 


M3 


Hills 


Spurs/Fingers 


Trails 


Clearings 


Buildings 


Compass 


M4 


Hills 


Fingers 


Ridge Lines 


Depressions 


Paved Roads 


Dirt Roads 


M5 


Compass 


Paved Roads 


Dirt Roads 


Hills 


Clearings 


Trails 


RWl 


Roads 


Trees 


Buildings 


Elevation 


Compass 


Hills 


RW2 


Dirt Roads 


Paved Roads 


Clearings 


Knolls 


Spurs 


Trees 


RW3 


Roads 


Trails 


Clearings 


Trees 


Hills 


Telephone 

Poles 


RW4 


Buildings 


Trail 


Poles 


Lakes 


Hills 


Depressions 


RW5 


Hills 


Ridge Lines 


Paved Roads 


Lakes 


Trees 


Public 

Buildings 


VEl 


Hills 


Spurs 


Tr^ls 


Buildings 


Ponds 


Compass 


VE2 


Hills 


Fingers 


Compass 


Buildings 


Roads 


Lakes 


VE3 


Roads 


Paths 


Electric Lines 


Ridge Lines 


Hills 


Lakes 


VE4 


Paved Roads 


Dirt Roads 


Hills 


Fingers 


Telephone 

Poles 


Buildings 


VE5 


Houses 


Dirt Roads 


Electric Lines 


Hills 


Lakes 


Ponds 


A1 


Paved Roads 


Dirt Roads 


Ridge Lines 


Lakes 


Hills 


Electric Lines 


PI 


Hills 


Ridge Lines 


Dirt Roads 


Paved Roads 


Spurs 


Clearings/Trees 


P2 


Dirt Roads 


Paved Roads 


Buildings 


Trails 


Rivers/Streams 


Hills 


P3 


Paved Roads 


Dirt Roads 


Public 

Buildings 


Houses 


Lakes 


Trees 


P4 


Dirt Roads 


Trails 


Paved Roads 


Shacks 


Towers 


Electric Lines 
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APPENDIX R ENVIRONMENT COMPARISONS 



1. GENERAL INFORMATION 

This appendix outlines the differences in the training conditions and participant 
results between the Banker and Goerger experiments. The training conditions include a 
non-real time model, real time model, real world exposure, and map only study. NA 
indicates characteristics that do not apply to a training condition due to the nature of the 
training medium. Results are broken down by experiment and by training condition. 
Results such as the Total Map Check Scores, Wheel Test Angular Differences and White 
Board Angular Differences are not directly comparable because of the different 
information included, means of measurement, or tests conducted in the separate 
experiments. 

2. ENVIRONMENT CHARACTERISTICS 

The following is a listing of characteristics and which training treatment 



implements each one. 



Characteristic 


Non-Real Time 
Model (Banker) 


Real Time Model 
(Goerger) 


Real World 


Map 


"You are here!" Designator (Constant) 


Yes 


No 


No 


NA 


"You are here!" Designator (On Demand) 


Yes 


Yes 


No 


NA 


1.5m to 2m Elevated View Point 


Yes 


Yes 


Yes 


No 


15m Elevated View Point 


No 


Yes 


No 


No 


Animals 


No 


No 


Yes 


NA 


Boundary Terrain 


No 


Yes 


Yes 


Yes 


Bushes 


Yes 


Yes 


Yes 


NA 


Clearings 


Yes 


Yes 


Yes 


Yes 


Compass 


No 


Yes 


Yes 


NA 


Continues Model 


No 


Yes 


Yes 


Yes 


Depressions 


Yes 


Yes 


Yes 


Yes 


Dirt Roads 


Yes 


Yes 


Yes 


Yes 


Draws 


Yes 


Yes 


Yes 


Yes 


Electric Lines 


No 


No 


Yes 


Yes 


Factory 


No 


No 


No 


No 


Flowers 


No 


No 


Yes 


No 


Foot Paths 


Yes 


Yes 


Yes 


Yes 


GrassAVeeds 


Yes 


No 


Yes 


No 


Hills 


Yes 


Yes 


Yes 


Yes 


Houses 


No 


No 


No 


No 


Interface Testing 


No 


Yes 


No 


No 


Interface Train-Up Phase 


No 


Yes 


No 


No 


Joystick Interface 


No 


Yes 


No 


No 


Keyboard Interface 


Yes 


Yes 


No 


No 


Knolls 


Yes 


Yes 


Yes 


Yes 


Lakes 


No 


No 


No 


No 
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Characteristics (continued) 


Non-Real Time 
Model (Banker) 


Real Time Model 
(Goer^er) 


Real World 


Map 


LOD Snapping/Popping 


No 


Yes 


No 


No 


Marsh Lands 


No 


No 


No 


No 


Mouse Interface 


Yes 


No 


No 


No 


Paved Roads 


Yes 


Yes 


Yes 


Yes 


Pedometer (Distance Traveled) 


No 


No 


Yes 


Yes 


People 


No 


No 


Yes 


No 


Pitch Viewpoint Up/Down 


No 


Yes 


Yes 


NA 


Pits/Fox Holes 


Yes 


No 


Yes 


Yes 


Ponds 


No 


No 


Yes 


No 


Power Lines 


No 


No 


Yes 


Yes 


Public Buildings 


No 


No 


No 


No 


Puddles 


No 


No 


No 


No 


Real Time 


No 


Yes 


Yes 


NA 


Realistic Control Point Markers 


No 


Yes 


Yes 


Yes 


Realistic Telephone Poles 


No 


Yes 


Yes 


Yes 


Ridgelines 


Yes 


Yes 


Yes 


Yes 


River 


No 


No 


No 


No 


Road Signs 


No 


No 


Yes 


No 


Rock Piles 


Yes 


Yes 


Yes 


Yes 


Rotate view to side while moving 


No 


Yes 


Yes 


NA 


Sand Bags 


No 


Yes 


Yes 


No 


Shacks 


Yes 


Yes 


Yes 


Yes 


Shadows 


No 


Yes 


Yes 


No 


Shallow Ditches 


Yes 


Yes 


Yes 


Yes 


Sound 


No 


No 


Yes 


No 


Spurs/FingCTs 


Yes 


Yes 


Yes 


Yes 


Stream 


No 


No 


No 


No 


Street Signs 


No 


No 


No 


No 


Swamps 


No 


No 


No 


No 


Telaportation 


Yes 


Yes 


No 


NA 


Telephone Poles 


Yes 


Yes 


Yes 


Yes 


Terrain Segregation 


Yes 


No 


Yes 


Yes 


The Sun 


No 


No 


Yes 


No 


Top Down View Point 


Yes 


Yes 


No 


Yes 


Towers 


No 


No 


Yes 


No 


Trails 


Yes 


Yes 


Yes 


Yes 


Trees 


Yes 


Yes 


Yes 


Yes 


Trenches 


Yes 


Yes 


Yes 


Yes 


Undergrowth 


Yes 


Yes 


Yes 


Yes 


Variable Speeds of Movement 


No 


Yes 


Yes 


NA 


Variable Weather 


No 


Yes 


Yes 


NA 


Wide Field of View 


No 


Yes 


Yes 


NA 



Table P.l. Training Environment Characteristics 
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3. PARTICIPANT GROUP RESULTS 



Behavior and Performance 
Differences 


Non-Real 
Time Model 
(Banker) 


Real Time 
Model 
( Goer^er) 


Real 

World 

(Banker) 


Real 

World 

(Goer^er) 


Map 

(Banker) 


Map 

(Goerger) 


Average Control Points Found 


8.000 


5.600 


7.400 


6.800 


6.800 


7.200 


Average Control Points Attempted 


8.400 


5.600 


7.600 


6.800 


7.400 


7.200 


Average Errors 


3.800 


5.800 


2.800 


4.800 


3.800 


4.800 


Average Eitots Per Control Point 
Attempted 


0.452 


1.034 


0.368 


0.788 


0.514 


0.678 


Average Errors Per CP # 4 


0.600 


2.400 


0.800 


1.600 


1.000 


1.200 


Distance Per Error (meters) 


435.789 


627.040 


470.429 


464.556 


345.684 


502.850 


Distance Per Errcr Per Control Point 
Attempted (meters) 


51.880 


131.984 


61.898 


80.046 


46.714 


83.036 


Average Map Checks 


7.000 


7.400 


4.000 


5.600 


5.000 


6.200 


Average Map Checks Per Control Point 
Attempted 


0.833 


1.194 


0.526 


0.778 


0.676 


0.816 


Average Compass Checks 


NA 


0.200 


NA 


0.600 


NA 


0.000 


Average Compass Checks Per Control 
Point Attempted 


NA 


0.032 


NA 


0.083 


NA 


0.000 


Average Map and Compass Checks 


0.000 


1.600 


0.000 


5.800 


5.800 


0.200 


Average Map and Compass Checks Per 
Control Point Attempted 


0.000 


0.258 


0.000 


0.806 


0.784 


0.026 


Average Out of Bounds 


NA 


0.800 


NA 


0.000 


NA 


0.400 


Average Out of Bounds Per Control 
Point Attempted 


NA 


0.129 


NA 


0.000 


NA 


0.053 


Average Reorientation by Monitor 


NA 


1.000 


NA 


0.600 


NA 


0.400 


Average Reorientation by Monitor Per 
Control Point Attempted 


NA 


0.161 


NA 


0.083 


NA 


0.053 


Average Map Check Score 


NA 


15.500 


NA 


17.400 


NA 


9.200 


Average Map Check Score Per Control 
Point Attempted 


NA 


3.104 


NA 


2.836 


NA 


1.424 


Average Wheel Test Angular 
Differential CP2 (Pointing Task) 


NA 


20.134 


NA 


27.668 


NA 


26.666 


Average Wheel Test Angular 
Differential CP4 (Pointing Task) 


NA 


31.890 


NA 


19.418 


NA 


23.918 


Average Wheel Test Angular 
Differential (Pointing Task) 


NA 


26.166 


NA 


27.500 


NA 


26.634 


Average White Board Test Angular 
Differential (Geo target Placement 
Task) 


NA 


21.376 


NA 


27.588 


NA 


22.227 


Average Unplanned Route Execution 


NA 


0.200 


NA 


0.400 


NA 


0.600 


Simulation Sickness 


No 


Yes 


No 


No 


No 


No 



Table P.2. Participant Results by Training Treatment 
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APPENDIX Q. NAVIGATION CYCLE 



1. GENERAL 

The phases of the navigation cycle are displayed in Figure Q.l. This diagram is a 
simplified flow chart depicting how an individual may conduct navigation through an 
environment. It is generic in nature to allow it to be applied to navigation of varied 
media and environments by any mode of locomotion. 




Figure Q. 1 . Navigation Cycle 
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The model takes into account many of the factors used in previous models which 
attempt to depict the process of wayfmding [JUL 97] [WICK 98]. Figure Q.l is similar 
in its purpose and design as the one currently under development and verification by 
Dr. Wickens [WICK 98]. However, this chart is more generic than Dr. Wickens’ and is 
designed to quickly identify the stage of navigation an individual is in during course 
execution. Using the data collected in this experiment in conjunction with a viable 
diagram outlining the task of navigation, we can gain a better understand of the process 
of navigation, determine where failures in navigation are most likely to occur, and create 
systems or provide training to correct those shortcomings. With proper training and 
evaluation tools, we can predict trouble areas during mission planning and preparation 
that can be corrected to assist in conducting more efficient maneuvers to target areas, 
reserving resources for other crucial requirements. 

The diagram separates the task of navigating into four distinct areas; route 
planning, route navigation, navigational aids, and error recovery. While navigating, all 
individuals will undergo route navigation. The phase of error recovery is a tributary 
phase that is utilized only if individuals feel or recognize they are no longer following 
their initially visualized or planned route. 

2. ROUTE PLANNING 

Before movement is conducted or after a navigational error has been recognized, 
most individuals fix their position and orientation, identify the intended goal, determine 
method of movement, and plane route to traverse the space between the current position 
and target position. These processes are grouped together and are known as route 
planning. To facilitate route planning, individuals seek information about the 
environment and their location within that environment from navigational aids (Appendix 
Q, Section 5). 

3. ROUTE NAVIGATION 

During route navigation, individuals break their movement down into three 
distinct areas, coarse movement, fine movement, and maintenance. Coarse movement is 
categorized as the general movement that occurs as individuals traverse indistinctive 
terrain in search of a linear catching feature or landmark which tells them they are in the 
general area of the target. Distances vary in length based on the limits of visibility and 
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target area size. The area covered during movement provides few cues which individuals 
focus their attention on. Individuals are concerned with directional signs [PASS 84] that 
teU them they are moving in the appropriate direction. They are also concerned with 
distinctive features or regional signs that wiU tell them they are on their planned route, in 
the target area, or have passed the target area. For example, if traveling from New York 
City to Chicago by car to watch a Chicago Bulls game, we are not concerned with the 
road signs teUing us that there is a sale at Johnson’s Lumber. However, we are concerned 
with signs telling us “Welcome to Ohio”, “Interstate 90 North 2 Miles”, and “Mississippi 
River”. The first two provide us with information that we are traveling in the right 
direction, directional sign, or that we are close to our target area, regional sign [PASS 
84]. The last sign tells use we have gone past our intended destination and we need to 
stop and plan a new route to Chicago. In a natural environment, directional signs for 
dismounted movement would be items such as the sun, moss on the north sides of a tree, 
the North Star (Polaris), or the flow of major rivers. Regional signs would be clearings, 
sand dunes, red wood groves, or villages. 

The fine movement phase is characterized by a more detailed search using more 
distinctive landmarks. During this phase, individuals pay more attention to their 
surroundings focusing on minor changes in the environment known as identification 
signs [PASS 84] that may lead them to the target. An example is when we reach the 
outer loop of the Chicago beltway, we start searching for specific exits and street names 
to take us to the stadium. As we get closer to the stadium we search for information 
about parking. Identification signs for this experiment would be the wooden shanks, 
pavilions, telephone poles, rock piles, and major trail intersections. Due to the random 
and relatively unstructured nature of natural environments, any unambiguous terrain 
feature can be used as an identification sign. 

Throughout the route navigation process, individuals regularly perform mental 
map maintenance. As individuals maneuver through their environment, they continually 
search for information from navigational aids (Appendix Q, Section 5) to confirm general 
location and proper movement heading. This information is known as reassurance signs 
and includes items such as mileage markers and exit numbers [PASS 84]. They are 
usually selected during the route planning phase and used as self-imposed checks during 
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route navigation to ensure individuals remain on their planned route. In a natural 
environment, reassurance signs are more ambiguous than those found in man-made 
environments. For this experiment’s environment, reassurance signs would be the 
number of trails crossed, trail intersections, high ground, low ground, or power lines. If 
there are differences between the physical or mental map and the information the 
individual perceives, the differences must be resolved. If there are no differences 
between the mental map, physical map, environment, and where individuals believe they 
are, movement continues toward the intended destination. Individuals freely move 
between coarse movement, fine movement, and maintenance based on the conplexity of 
the route and mission. 

Resolving minor differences can be as simple as updating one’s mental map to 
include the new Pizza Hut built off of Exit 68 or the bum barrel south of Control 
Point #2. When they are unable to correctly resolve differences between their maps and 
the environment because they failed to recognize a landmark, they become confused. 
This is a phase where individuals attempt to resolve major differences in the mental and 
physical maps with their surroundings. If they cannot determine if they are lost or if they 
need to update their mental map, individuals remain in a data collection loop attempting 
to gather detailed information from navigational aids to make that determination. At the 
conclusion of this phase one must return to route navigation or move to error recovery. 

4. ERROR RECOVERY 

Once individuals have identified that they are lost, they enter the error recovery 
phase where they determine what their mistake may have been. Determining their error 
assists the individual with coming to grips with the situation and provides an indication of 
a potential recovery strategy. After recognizing the error committed, individuals move to 
the route planning phase. 

5. NAVIGATIONAL AIDS 

Navigational aids are used to provide individuals with information to update their 
mental maps, determine position and orientation, plan routes, and execute movement. 
They are utilized in accordance with an individual’s experience, training, and confidence 
level. The aids are placed in one of two subcategories based on the type of information 
they provide, static imagery or dynamic imagery. Static imagery renders propositional 
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information about the environment and the individual’s place within the environment. 
This imagery is provided from items that furnish positional information (GPS, map, 
signs), orientation information (compass, map, sun), or stationary target information 
(map, pictures). Dynamic imagery supplies temporal information about the environment 
and the individual’s place within the environment. This imagery is provided from items 
that furnish disambiguating or continuous information. This information is derived from 
the environment, VEs, videos, and other active sources of information. This information 
is used to disambiguate positional information. 
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